[Bug] v3.1.0 model-chain fallback never switches the upstream provider — all retry attempts are re-sent to the dead primary

[Bug] v3.1.0 model-chain fallback never switches the upstream provider — all retry attempts are re-sent to the dead primary

Quick summary: After upgrading 3.0.22 → 3.1.0, when the primary model dies with HTTP 403 (quota exhausted), the model-chain fallback executes (trace shows 4 attempts with correct backoff) but every attempt is still physically sent to the primary's upstream with the primary's model — the chain never actually leaves the dead provider. Downgrading back to 3.0.22 restores correct behavior, so this is a 3.1.0 gateway regression.

Environment

• Claude Code Router v3.1.0 (Windows, electron build Claude-Code-Router_3.1.0.exe), upgraded from v3.0.22
• Gateway mode: local OpenAI-compatible gateway at http://127.0.0.1:3456/v1
• Also verified on v3.0.22 after downgrade: fallback works there, same config

Configuration

A router rule rewrites an alias to a primary model, with a model-chain fallback:

{
  "name": "proxy-harness-planner",
  "type": "condition",
  "condition": { "left": "request.body.model", "operator": "==", "right": "harness-planner" },
  "rewrite": { "key": "request.body.model", "operation": "set", "value": "k3" },
  "fallback": {
    "mode": "model-chain",
    "models": [
      "智谱 AI (国内) - Coding Plan/glm-5.3",
      "公司api/GLM-5.3",
      "公司api/gpt-5.6-sol"
    ],
    "retryCount": 1
  }
}

Providers involved: KimiCode (k3), zhipu bigmodel coding plan (glm-5.3), a company OpenAI-compatible endpoint (GLM-5.3, gpt-5.6-sol). All protocols are openai_chat_completions.

Steps to Reproduce

1. On v3.1.0, configure the rule above (primary k3, fallback chain to other providers).
2. Exhaust the primary's quota so k3 returns HTTP 403 access_terminated_error ("weekly usage limit"). Any hard-failing primary works.
3. Send a streaming chat completion through the gateway with model harness-planner:
   bash复制

   curl -N http://127.0.0.1:3456/v1/chat/completions \
     -H "Authorization: Bearer <gateway-key>" -H "Content-Type: application/json" \
     -d '{"model":"harness-planner","max_tokens":64,"stream":true,"messages":[{"role":"user","content":"hi"}]}'

4. Observe the failure. The route trace in request-logs.sqlite claims 4 attempts across 4 different providers, but the actual outbound traffic tells another story (see evidence).

Expected Behavior

When the primary returns 403, the fallback chain should dispatch attempt 2 to zhipu glm-5.3, attempt 3 to 公司api/GLM-5.3, etc. — i.e. each retry must target a different upstream with the fallback's model name in the request body. This is exactly what v3.0.22 does: after the downgrade, the same request resolves via ai---coding-plan::openai_chat_completions/glm-5.3 with route_attempt_count=2, HTTP 200.

Actual Behavior

• Client receives the primary's error: HTTP 403 with the KimiCode quota body, and the gateway-level error summary lists only one target provider ("target_providers":["openai"], "target_provider_names":["kimicode::openai_chat_completions"], attempts:[...one kimicode attempt...]), even though the per-hop trace shows 4 attempts "prepared" against the fallback targets.
• The upstream request/response spool (request-log-bodies) proves every retry was physically sent to the dead primary: for one probe request I found 3 pairs of {"model":"k3", ...} request bodies each answered by the identical Kimi quota-403 body (timestamps match the 1s/2s/4s backoff). Zero requests were spooled to zhipu or the company endpoint during that window.
• Meanwhile, a different rule whose primary is zhipu glm-5.3 (no fallback needed) succeeds at the same moment through the same gateway — so the fallback candidates themselves were healthy; they were simply never contacted.

Evidence (3.1.0 failure vs 3.0.22 success)

Route trace of one failing request (from request_route_traces, v3.1.0):

attempt 1  kimicode::openai_chat_completions/k3                → 403 (quota, expected)
attempt 2  ai---coding-plan::openai_chat_completions/glm-5.3   → 403   ← trace claims zhipu, but no zhipu request ever hit the wire
attempt 3  codex::openai_chat_completions/GLM-5.3              → 403   ← same, no outbound request
attempt 4  codex::openai_chat_completions/gpt-5.6-sol          → 403   ← same
final client response body = KimiCode weekly-quota 403 (the primary's error)

Spooled upstream bodies during that same request (v3.1.0): all "model":"k3", all responses = Kimi quota 403.

After downgrading to v3.0.22 with identical config:

request id 107667: harness-planner → attempt 1 k3 (403) → attempt 2 ai---coding-plan/glm-5.3 → 200 OK   (route_attempt_count=2)
request id 107669: harness-max     → attempt 1 k3 (403) → attempt 2 ai---coding-plan/glm-5.3 → 200 OK   (route_attempt_count=2)

Historical baseline on 3.0.22: 1,200+ successful fallback switches in the request log the day before the upgrade; the very first chain-wide failure occurred 7 minutes after the 3.1.0 install, and the bug disappeared immediately after the rollback.

Root Cause (suspected)

In the fallback execution loop, the per-attempt dispatch headers/URL appear to be derived from the original (rule-routed) request — e.g. the x-target-providers / x-ccr-routed-model headers set for the primary keep pointing at the primary provider, and the gateway re-dispatches to it regardless of the per-attempt model. The trace records the planned target, so the UI/logs look correct while the wire traffic never leaves the dead provider. (Possible interaction: v3.1.0 gateway dispatcher changes such as PR #1699 / #1715.)

Impact

Any user whose primary provider hits a quota/auth hard failure gets a total outage through the gateway even when healthy fallback providers are configured — the chain silently retries the dead upstream 4 times (also burning 1+2+4s of backoff) and returns the primary's error. This makes the entire model-chain fallback feature a no-op for rule-based routes on 3.1.0.

Workaround

Downgrade to v3.0.22 (verified working). Alternatively, rewrite the rule's primary to a healthy model so fallback never triggers.

Happy to provide the full trace_json, spool file samples, or config exports if useful.