Chat template breaks KV-cache reuse when enable_thinking=false

Problem

When enable_thinking=false, the Qwen3/Qwen3.5 chat template inserts <think>\n\n</think>\n\n only for the last assistant turn (generation prompt), but not for previous assistant turns in the conversation history.

This means the token sequence from request N is not a prefix of request N+1, breaking KV-cache reuse in every inference server (mlx-lm, llama.cpp, vLLM, LM Studio, etc.).

Example

Request 1 (system + user):

<|im_start|>system\nYou are helpful.<|im_end|>\n<|im_start|>user\nHi<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n

→ Server caches this token sequence.

Request 2 (system + user + assistant history + new user):

<|im_start|>system\nYou are helpful.<|im_end|>\n<|im_start|>user\nHi<|im_end|>\n<|im_start|>assistant\nHello!<|im_end|>\n...

→ The assistant turn from request 1 is now rendered without <think>\n\n</think>\n\n tags → token prefix mismatch → cache miss.

Root Cause

In the Jinja template (tokenizer_config.json), the assistant message rendering logic:

{%- if loop.index0 > ns.last_query_index %}
    {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
{%- else %}
    {{- '<|im_start|>' + message.role + '\n' + content }}
{%- endif %}

The else branch (history messages before the last query) does not add <think>\n\n</think>\n\n when enable_thinking=false. Only the add_generation_prompt section at the bottom does.

Fix

Add an enable_thinking is false check to the history assistant messages as well:

{%- if loop.index0 > ns.last_query_index %}
    {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
{%- elif enable_thinking is defined and enable_thinking is false %}
    {{- '<|im_start|>' + message.role + '\n<think>\n\n</think>\n\n' + content }}
{%- else %}
    {{- '<|im_start|>' + message.role + '\n' + content }}
{%- endif %}

This ensures consistent tokenization across requests, enabling proper KV-cache prefix matching.

Impact

Tested on Apple M2 Max with nightmedia/Qwen3.5-35B-A3B-Text-qx64-hi-mlx via mlx-lm 0.30.7:

This is a ~90x speedup for multi-turn conversations. Without the fix, every turn reprocesses the entire prompt from scratch.

Affects

• All Qwen3 models (Qwen3-235B-A22B, Qwen3-30B-A3B, Qwen3-8B, etc.)
• All Qwen3.5 models (Qwen3.5-35B-A3B, Qwen3.5-122B-A10B, etc.)
• Every inference server with prompt caching (mlx-lm, llama.cpp, vLLM, SGLang, LM Studio)

Related

• Qwen3-1.7B HuggingFace Discussion #9
• mlx-lm Issue #903
• llama.cpp Issue #18497