MixinSession failover concatenates responses after partial output

Author: Whxuan0701Created Sep 8, 2026Updated Sep 8, 2026

Problem

MixinSession.raw_ask() can fail over after the current backend has already emitted visible output. The next backend then starts the same answer from scratch, so one assistant turn becomes a concatenation of responses from different models.

Example observed on main@f6e5657 with deterministic fake sessions:

A-part
!!!Error: upstream reset
B-complete

Both A and B were called. For users, this can produce duplicated prose, contradictory conclusions, or an invalid mixed tool call.

Reproduction

Configure two MixinSession backends and let the first generator yield:

python
["A-part", "!!!Error: upstream reset"]

Let the backup yield:

python
["B-complete"]

Collect one MixinSession.raw_ask() response. Current output contains all three chunks, and both backends receive the request.

Root cause

raw_ask() tracks whether anything has been yielded, but that flag is used only to suppress leading error chunks. Any terminal !!!Error: remains eligible for the same retry loop even after earlier chunks have already been committed to the caller.

Expected behavior

The first visible chunk is a commit point:

  • Failure before visible output may transparently fail over.
  • Failure after visible output should surface on the current response without appending a second backend's response.
  • The backup may be selected for the next request.
  • The current generator should still be drained so its returned content blocks and transport cleanup are preserved.

Suggested direction

Track a partial-output failure separately from a pre-output failure. Once committed, yield the terminal error once, drain the generator without forwarding additional chunks, return its content blocks, and rotate the preferred session only for the next call.

This is separate from #753/#767, which decide whether an SSE failure is retryable inside one transport session. This issue is about cross-backend failover semantics after output has already escaped the routing boundary.