History compression permanently stalls when utility model fails, causing silent empty final responses on saturated chats

Summary

When the utility model fails during history compression (quota exhaustion, network errors, or empty content), compression permanently stalls and never retries successfully. The chat context then stays saturated, the chat model returns empty/unusable content on subsequent turns, and no final response is ever delivered — with no error shown to the user. We hit this in production on two long-running chats (~103k token contexts, 4500+ iterations); the symptom is exactly the widely-reported "Agent Zero stops replying mid-conversation" behavior.

Log signature:

WARNING: History compression stalled
History compression could not reduce the prompt history further. Tokens before: 140261; after: 140261.
...
Agent stopped after 5 consecutive unusable model responses to prevent further API charges.

Reproduction

1. Run a chat long enough to trigger repeated history compression (100k+ tokens of history).
2. Let the utility model fail — e.g. exhausted free-tier quota (User has no quota left), a provider outage, or a provider that returns empty content under load.
3. Observe: compression logs "stalled" with identical before/after token counts and never makes progress again.
4. Every subsequent turn: the chat model receives an over-limit context, the provider returns empty/unusable content, the repair loop gives up after 5 attempts, and the user gets no answer at all.

The failure is intermittent at first (only when the utility model errors), then effectively permanent — the context never shrinks back below the limit.

Root cause

python/helpers/history.py (current development):

1. All compression paths beyond cheap large-message truncation depend on LLM summarization.

   • Topic.compress() (line ~257) tries compress_large_messages() first (no LLM), then compress_attention().
   • compress_attention() (line ~263) calls summarize_messages() → agent.call_utility_model().
   • History.compress_topics() (line ~576) → topic-to-bulk merging → Bulk.summarize() → agent.call_utility_model().
   • History.compress_bulks() (line ~600) → merge_bulks_by() → Bulk.summarize() again.

2. None of these summarize calls handle failure. When call_utility_model() raises (litellm APIError/PermissionDeniedError etc.), the exception aborts the whole cascade. OrganizeHistoryWait (extensions/python/message_loop_prompts_before/_90_organize_history_wait.py) then logs "History compression stalled" and breaks out of its retry loop — and because the token count never decreased, is_over_limit() stays true forever. There is no non-LLM fallback, so a dead utility model means compression can never succeed again for that chat.

3. Secondary bug: History.compress_bulks() does self.bulks.pop(0) after merge_bulks_by() returns False — but merge_bulks_by() also returns False when self.bulks is empty, so pop(0) raises IndexError on an empty list (only reachable when the first bulk-merge path fails and the exception is caught upstream).

Proposed fix

Minimal, failure-path-only patch (no behavior change on the happy path):

1. Topic.summarize_messages() and Bulk.summarize(): wrap the call_utility_model() call in try/except Exception and fall back to a deterministic non-LLM _truncate_summary() (head+tail truncation with an explicit marker) so compression always makes progress.
2. History.compress_bulks(): return False when self.bulks is empty instead of pop(0) on an empty list.

def _truncate_summary(text: str, max_chars: int = 2000) -> str:
    text = (text or "").strip()
    if len(text) <= max_chars:
        return text or "(no content)"
    head = int(max_chars * 0.6)
    tail = max_chars - head
    return (
        text[:head]
        + "\n[... content truncated by emergency fallback compression ...]\n"
        + text[-tail:]
    )

A more conservative alternative (used by closed PR #1114 for the same area) is to only swallow the exception and log — but that does not fix the stall: compression still makes no progress, so the chat stays saturated. A fallback that guarantees progress is the minimum effective fix.

Before / after

Verification (local)

• Simulated a permanently failing utility model against History.current.compress(): tokens reduced 40,081 → 24,108 in one pass (previously: exception → zero progress).
• Bulk.summarize() fallback produces a marked truncated summary instead of raising.
• All 9 existing related tests pass (tests/test_history.py, tests/test_history_compression_wait.py, tests/test_chat_compaction.py); the failing-utility-model path is covered by the functional check above.

Environment

• Agent Zero: commit 6a6cecff (2026-08-27 checkout), Docker deployment
• Chat model: GLM Flash preset via a Venice-style proxy; utility model on the same proxy (quota failures)
• Contexts observed: ~103–140k tokens, 35 unsummarized topics / 113k tokens stuck

Note: PR #1114 (closed, unmerged) touched this area with a swallow-and-log approach, which alone does not restore compression progress.