AI Gateway: zai/glm-5.3-flash via Baseten bills the final answer but streams no content when tools are present

Description

Through AI Gateway, zai/glm-5.3-flash served by Baseten sometimes generates and bills the final answer but never streams it. The response finishes stop, the reasoning arrives, and usage.outputTokenDetails.textTokens is > 0, yet no chunk carries any content. Callers get text: "".

In our runs it happens only on the Baseten route, and only when the request includes tools and the model reasons before answering. Other providers for the same model and the same request return the answer.

Environment

• ai 7.0.30, @ai-sdk/gateway 4.0.22, Node 22
• Model zai/glm-5.3-flash, routed by the gateway to baseten (about 97% of our traffic for this model lands there)

Reproduction

streamText (and generateText, same result) with a short user message ("Reply with only the word PONG"), a system prompt, and a set of tools. The provider is pinned with providerOptions.gateway.only. includeRawChunks: true shows the upstream chunks.

We ran 480 calls. The table counts calls where the model reasoned before answering; "empty" means textTokens > 0 but no content delivered. Capacity 429s are excluded.

• Baseten with tools vs Baseten without: p = 6e-7 (Fisher).
• Baseten with tools vs zai with tools: p = 5e-6.
• No effect from caching, reasoning low/high, a session header, or generateText vs streamText.
• reasoning: "none" mostly stops the reasoning, but a call that still reasoned came back empty.
• In calls that did succeed, the raw content matched the SDK text exactly, so the SDK is not dropping anything. The content is missing from the gateway's raw chunks too.

Raw chunks

Empty call, gen_01M2PGPTV1XWEYVEH2KAAPT58C. The billed answer never appears:

{"delta":{"content":"","role":"assistant","reasoning_content":"P"}}   completion_tokens=1 reasoning_tokens=1
{"delta":{"content":"","reasoning_content":"ONG"}}                   completion_tokens=5 reasoning_tokens=2
{"delta":{},"finish_reason":"stop"}                                  completion_tokens=6 reasoning_tokens=2
→ usage {total: 6, text: 4, reasoning: 2}, text ""

Successful call on Baseten with the same token progression, gen_01M2PGGTDJ7KHXM9BJWRZ9448X:

{"delta":{"content":"","role":"assistant","reasoning_content":"P"}}   completion_tokens=1 reasoning_tokens=1
{"delta":{"content":"","reasoning_content":"ONG"}}                   completion_tokens=5 reasoning_tokens=2
{"delta":{"content":"PONG"}}                                         ← this chunk is missing in the empty call
{"delta":{},"finish_reason":"stop"}                                  completion_tokens=6 reasoning_tokens=2

Another successful call in the same pinned-Baseten batch: gen_01M2PGQ6CDYZTMNW85SYWQ3PA5.

In production this loses the final message of a chat turn. It happens in roughly 0.4% of our GLM turns, and much more often for very short final answers.

Possibly related

• sgl-project/sglang#37645 (open), fix in sgl-project/sglang#37644: the glm45 reasoning parser returns the answer as reasoning_content with empty content and finish stop. That is a similar symptom. We don't know whether Baseten's GLM deployment uses that parser.
• #19866: an earlier Baseten-route failure for zai/glm-5.3-flash (503s).
• #17274: reasoning tokens billed with no content returned, on Google.

Ask

Could the Baseten deployment of the GLM models (and other reasoning models served there) be checked for how reasoning and content are split when tools are present? Until then, is there a recommended way to route one model away from a single provider without hard-coding an only allow-list?