glm53: KV prefix reuse lost after bare tool-call turns — template writes "\n<tool_call>", the model does not

Summary

On the glm53 engine, a multi-turn tool conversation loses its cached KV prefix on the turn after any assistant turn that is a bare tool call (no text). The cause is one token: render_chat_glm53 follows chat_template.jinja and writes "\n<tool_call>", but the model emits </think><tool_call> with no newline. The gate in glm53.c (slot_shared(...) >= cached, ~line 2483) is all-or-nothing, so the whole prefix is discarded and the turn re-prefills from scratch.

Plain text turns are unaffected (REUSE 5 22 33 on a two-turn chat). Text-then-tool-call turns mostly reuse, because the model's own trailing newline is stripped and re-added, and the tokens line up by accident.

Environment

• Strix Halo (Ryzen AI MAX+ 395, 128 GB), Arch, main @ fd93c41
• GLM-5.3-Flash int4 (glm53_i4), coli --auto-tier --ngen 4096 serve, GLM53_MAXT=65536, KV_SLOTS default (1)
• Prefill ≈ decode ≈ 2.3 tok/s here, so a lost prefix on a 3k-token agent history costs 20+ minutes

Repro (two requests)

Turn 1:

{"model":"glm-5.3-flash-colibri","max_tokens":300,
 "messages":[{"role":"user","content":"List the files in /workspace. Use the tool."}],
 "tools":[{"type":"function","function":{"name":"bash","description":"Run a shell command",
  "parameters":{"type":"object","properties":{"command":{"type":"string"}},"required":["command"]}}}]}

Model output (COLI_DEBUG=1 tee): </think><tool_call>bash<arg_key>command</arg_key><arg_value>ls /workspace</arg_value></tool_call> — 12 tokens, finish_reason: tool_calls.

Turn 2 = turn 1 + the returned assistant message (content: null, tool_calls) + {"role":"tool","content":"sample.csv\n"}.

Rendered prompt for turn 2 (COLI_DEBUG=2):

...<|assistant|><think></think>\n<tool_call>bash<arg_key>command</arg_key><arg_value>ls /workspace</arg_value></tool_call>\n<|observation|>...

GLM53_VERBOSE=1:

REUSE 1 0 170
REUSE 2 0 192      <- expected ~182 shared

A temporary probe placed just before the gate:

PROBE 2 cached=182 slot_n=182 total=192 common=171 first_diff@171 slot=154843 new=198

Position 171: the slot holds 154843 (<tool_call>), the re-rendered prompt holds 198 ("\n").

Ruled out on the way: special-token splitting (tok_encode handles it), non-canonical BPE (colibri's tok.h and HF tokenizers give the same 12 ids), server STOP/CANCEL (engine reads control frames only between requests), the <|observation|> text stop filter (explicit stop → same result).

Workaround

vrinek/colibri@8b972ce767ea (branch glm53-toolcall-prefix-reuse): when the assistant content is empty, lstrip("\n") the rendered calls. Same repro afterwards:

REUSE 2 182 191    turn 2: 11 s (was 70 s)

It deliberately departs from the template, so tests/test_glm53_chat_template.py fails on the "chiamata e risultato" case at exactly that newline. I have not changed the test.

Where a real fix might live

1. Template — the whitespace comes from {% if m.tool_calls %} / {% for %} lines without - trimming in zai-org's chat_template.jinja; the model does not reproduce it. Upstream fix there makes colibri's byte-for-byte parity correct automatically.
2. Server — keep the model's raw emitted text per completion (keyed by the call_… ids the server already mints) and re-render assistant turns from it when the client echoes those ids back. Exact match, no template deviation for anything else.
3. Engine — snapshot the session (incl. the KDA recurrent state) at the prompt/generation boundary, and on a mismatch inside the generated span rewind to that snapshot instead of resetting. Turn 2 would then reuse the 170 prompt tokens regardless of how the client re-renders the reply. Robust to every client-side drift, at the cost of one extra state copy per slot.

Happy to turn any of these into a PR.