qwen3.8-max (Model Studio Token Plan): sustained 1-4.5 min per-turn latency in long agentic tool-calling sessions forced mid-task provider switch

Summary

Sustained high latency on qwen3.8-max (Model Studio Token Plan endpoint) during a long agentic session with frequent tool calls: median ~53 s per turn, p90 ~200 s, worst ~279 s for turns whose output was only a small tool-call payload. The operator had to switch to another model/provider mid-task to get the work done.

Environment

• Client: Hermes Agent (open-source desktop agent framework), OpenAI-compatible chat-completions integration with tool calling, streaming enabled.
• Model: qwen3.8-max via Alibaba Cloud Model Studio Token Plan subscription.
• Endpoint host: token-plan.ap-southeast-1.maas.aliyuncs.com (/compatible-mode/v1).
• Workload: production-operations session — many sequential tool calls, growing multi-hundred-k-token context, prompt caching active.

Measured behavior (client-side message log)

One continuous session, ~113 minutes wall clock (23:21 → 01:15 local, UTC+03:00), 53 API calls, ~345k input tokens, ~6.4M cache-read tokens, ~162k output tokens.

Per-turn latency (tool result submitted → completed assistant response):

Examples of the worst turns (time, latency, output):

00:20:16  279s  tool_calls, ~0 chars of prose
23:56:37  271s  tool_calls, ~0 chars of prose
00:24:09  230s  tool_calls, ~0 chars of prose
00:07:15  230s  final answer, 4126 chars
00:03:22  226s  tool_calls, ~0 chars of prose
23:42:22  207s  tool_calls, ~0 chars of prose

Most slow turns emitted only a small tool-call payload (a few hundred characters), so output length does not explain the delay. Several turns were manually interrupted by the operator after 75+ seconds of silence.

Impact

This was time-sensitive production diagnostics. Because responses regularly took 1–4.5 minutes, the operator switched to a different model/provider mid-task to finish the work.

Request

Please investigate sustained latency for qwen3.8-max on the Token Plan endpoint under long-context agentic workloads with frequent tool calls. Happy to provide additional client-side timing traces on request.