Kimi K2 tool-call parsing fails when the model emits functions_<name>_<n> ids; failure surfaces as an error chunk containing the raw tool-call markup

Summary

When serving moonshotai/Kimi-K2.7-Code (MLX, two-node cluster), tool calls fail whenever the model emits a tool-call id in the form functions_<name>_<n> rather than the canonical functions.<name>:<n>. _parse_kimi_tool_calls rejects the id, parse_tool_calls turns the failure into an ErrorChunk, and the client receives:

• streaming: data: {"error": {"message": "<|tool_calls_section_begin|><|tool_call_begin|>functions_get_weather_1<|tool_call_argument_begin|>{...}<|tool_call_end|><|tool_calls_section_end|>", ...}} — the openai Python client surfaces this as APIError with the raw markup as the message
• non-streaming: a 500 with a non-JSON body (collect_chat_response raises ValueError)

The model emits the non-canonical form when it sees one in the conversation history: the K2 chat template renders tool_call.id verbatim into the prompt, and common clients (observed with Kimi Code CLI via a LiteLLM proxy) hand ids back sanitised to [A-Za-z0-9_-]. So in practice the first tool call in a session succeeds and later ones fail deterministically.

Where

Checked against main on 31 Aug 2026:

• src/exo/worker/engines/mlx/utils_mlx.py — _parse_kimi_tool_calls: _func_name_regex requires :\d+ immediately before <|tool_call_argument_begin|>, so functions_get_weather_1<|tool_call_argument_begin|> raises ValueError("No tool call found.").
• src/exo/worker/runner/llm_inference/tool_parsers.py — make_mlx_parser swallows the exception and returns None.
• src/exo/worker/runner/llm_inference/model_output_parsers.py — parse_tool_calls: on parsed is None it yields response.model_copy(update={"text": combined, "finish_reason": "error"}); map_responses_to_chunks converts that to ErrorChunk(error_message=response.text).
• src/exo/api/adapters/chat_completions.py — generate_chat_stream emits the ErrorChunk as an SSE error frame; collect_chat_response raises ValueError(error_message).

Repro

Two-turn request whose history contains a sanitised id. Point BASE at the cluster's OpenAI-compatible endpoint.

curl -s -w "\nHTTP %{http_code}\n" -X POST "$BASE/v1/chat/completions" \
  -H "Content-Type: application/json" -d '{
  "model": "moonshotai/Kimi-K2.7-Code", "stream": false, "max_tokens": 2000, "tool_choice": "auto",
  "tools": [{"type": "function", "function": {"name": "get_weather",
    "description": "Return weather for a named city.",
    "parameters": {"type": "object", "additionalProperties": false,
      "properties": {"city": {"type": "string"}}, "required": ["city"]}}}],
  "messages": [
    {"role": "user", "content": "Call get_weather exactly once for Adelaide, Australia. Do not answer in prose; use the supplied function."},
    {"role": "assistant", "content": null, "tool_calls": [{"id": "functions_get_weather_0", "type": "function",
      "function": {"name": "get_weather", "arguments": "{\"city\": \"Adelaide, Australia\"}"}}]},
    {"role": "tool", "tool_call_id": "functions_get_weather_0", "content": "{\"city\": \"Adelaide, Australia\", \"temp_c\": 17, \"conditions\": \"cloudy\"}"},
    {"role": "user", "content": "Thanks. Now call get_weather exactly once for Melbourne, Australia. Do not answer in prose; use the supplied function."}
  ]}'

Observed: HTTP 500. Changing the two occurrences of functions_get_weather_0 to functions.get_weather:0 and nothing else → HTTP 200 with a structured tool_calls entry functions.get_weather:1. Same result with "stream": true (error frame vs. a proper tool_calls delta).

Suggested changes

1. Accept the sanitised id variant in _parse_kimi_tool_calls, e.g. match ^\s*((?:functions[._])?(.+?)[:_](\d+))\s*<\|tool_call_argument_begin\|>. The trailing [_:]\d+ is unambiguous even when the function name contains underscores. Optionally normalise the returned id to functions.<name>:<n> so clients get the canonical form.
2. Degrade instead of erroring on parse failure in parse_tool_calls: yield the accumulated text as normal content (finish_reason stop/length) rather than finish_reason="error". A parser miss is recoverable by the client; an error chunk whose message is the model output is not, and it hides a perfectly good completion behind a 500.
3. (Optional) Normalise assistant tool_calls[].id in the incoming history to canonical form before applying the chat template, which prevents the model from learning the wrong form in the first place. This is what I've done in a proxy in front of Exo as a workaround and it resolved the issue completely across multi-step agent runs.

Happy to turn (1) and (2) into a PR if the approach is agreeable.

Environment

• exo main as of 2026-08-31, two Mac Studio M3 Ultra 512 GB, MLX ring
• Model: moonshotai/Kimi-K2.7-Code
• Client path: Kimi Code CLI → LiteLLM proxy (openai Python client 3.x) → Exo