Measurement: Qwen3-Coder-30B-A3B and instruction-injection via third-party documentation

Hello Qwen team,

This is a courtesy note about published research, not a vulnerability report — no embargo, no bounty request, and nothing here is a zero-day. It measures a known, unfixed class (prompt injection). I am posting publicly because I could not find a security contact for Qwen (security.txt is absent from qwen.ai, alibabacloud.com and alibabagroup.com); if a private channel exists, tell me and I will use it in future.

What I measured. I am researching whether coding agents can safely consume documentation harvested from third-party libraries. I planted an instruction inside a library's "documentation", asked an agent to write a small function using that library, and varied only where that text was placed: (A) as authoritative guidance, (B) quoted explicitly as untrusted third-party data with an instruction not to follow directives inside it, (C) in the system prompt. Compliance = the model acted on the planted instruction. Payloads were inert (a fake analytics call, an endpoint that was unregistered and resolved nowhere when the tests ran, a marker string).

Result for Qwen3-Coder-30B-A3B-Instruct (2026-08-21, MLX 4-bit via LM Studio):

run	A instruction	B data-framed	C system
core (N=3, 3 payloads)	6/9	8/9	not run
extended (N=2, 6 payloads)	9/12	9/12	7/12
qwen3.6-27b (run via LM Studio)	2/9	0/9	not run

The reason I am flagging it: this was the only model I measured where framing the text as untrusted data showed no protective effect — arm B matched or exceeded arm A (6/9 → 8/9 core, 9/12 → 9/12 extended). Every other agent tested dropped substantially or to zero under that framing. My working interpretation is that strong instruction-following — the thing that makes a coder model good — is what carries the injected instruction through the framing. That is an interpretation, not something the experiment establishes.

Qwen3.6-27B, measured separately, behaved unremarkably — 2/9 unframed and 0/9 framed as data, which is among the better results in the set. The observation above is about Qwen3-Coder specifically and does not carry across to it.

Limitations, stated plainly. Small N (2–3 per cell), one prompt template, one task domain (a Kotlin date-formatting function), 4-bit quantization, and compliance scored by string match on an inert marker with a hand-written scorer. Quantization and chat-template handling are both plausible confounders I have not ruled out. Treat these as single measurements, not as a characterization of the model; they are version- and date-stamped for that reason.

The finding I consider primary is architectural, not about your model. Across every agent tested, putting third-party text in the instruction or system channel defeats the "treat this as untrusted data" mitigation. That is a warning to people building documentation-harvesting tools — which includes me. This research is a negative result about my own design.

If the method is flawed, say so and I will correct or withdraw the numbers — the records are versioned and I will amend them after publication. If an unquantised or differently-templated run would give a fairer picture, tell me and I will run it and publish the corrected figure.

Method, payloads, raw transcripts and a runnable reproduction kit: experiments/test0/measurement/injection/ at https://github.com/dependencyskills/dependencyskills.

— Brill Pappin Dependency Skills · github.com/dependencyskills/dependencyskills