#3178 [Bug] _trim_messages(): turn-count check fires before token-budget check, causing unnecessary summary LLM calls

Self check

I'm on the latest version and searched existing issues (incl. closed) — no duplicate.

Environment

Version: v2.1.9 ( latest main )
OS: macOS 15 (also reproducible on Linux — the behavior is code-logic-dependent, not OS-dependent)
Python: 3.12
Install: source
Model & channel: any (the issue is in the context-trimming layer, independent of LLM provider)

What happened?

What I observed

_trim_messages() (agent/protocol/agent_stream.py:2454) runs a two-stage trimming pipeline:

Step 2 (line 2476): Turn-count check
  → if len(turns) > max_context_turns (30):
        remove first half + fire summary LLM call

Step 3 (line 2502): Token-budget check
  → if estimated_tokens > available_budget:
        tiered strategy (compress or discard)

Step 2 fires before Step 3, which means:

A conversation with 31+ short turns (e.g. 31 quick ls calls, each ~50 tokens) triggers Step 2's "remove half + summary LLM call", even though the total token count (~1,550 tokens) is well under the model's context budget.

This wastes an unnecessary LLM API call (the summary injection via _build_context_summary_callback()) on every such turn.

Root cause

The turn-count check (Step 2) and the token-budget check (Step 3) are independent triggers that run sequentially, not a unified budget-aware strategy:

# agent/protocol/agent_stream.py:2476-2486
if len(turns) > self.max_context_turns:
    removed_count = len(turns) // 2          # ← always removes exactly half
    keep_count = len(turns) - removed_count
    discarded_turns = turns[:removed_count]
    turns = turns[-keep_count:]

    # Flush to daily memory + inject context summary (single async LLM call)
    if self.agent.memory_manager:
        # ... fires _build_context_summary_callback → LLM call

Then Step 3 independently checks token budget:

# agent/protocol/agent_stream.py:2502-2530
context_window = self.agent._get_model_context_window()
output_reserve = self.agent._get_output_reserve_tokens()
input_ceiling = max(1, context_window - output_reserve)
# ...
current_tokens = sum(self._estimate_turn_tokens(turn) for turn in turns)

if current_tokens + system_tokens <= max_tokens:
    # Under budget → reconstruct and return (but Step 2 already trimmed!)

Two problems:

Step 2 doesn't know about Step 3's budget. It fires purely on turn count, regardless of whether the token budget is already satisfied.
Both steps use "remove half" rather than removing only the minimum turns needed. If 31 turns total 200K tokens and the budget is 50K, removing half still leaves ~100K (over budget), requiring another pass.

Additionally, _smart_compact_to_budget() (line 2324, the reactive overflow handler) repeats the same "discard half" strategy in a loop until the context fits — but this only fires after the API has already returned an overflow error.

Proposed change (minimal)

Merge Step 2 and Step 3 into a single token-budget-first pass: walk turns from newest to oldest, accumulate estimated tokens, keep the longest suffix that fits within the budget, and use max_context_turns as a safety net.

def _token_budget_trim(self, turns: list, budget: int) -> tuple:
    """From newest to oldest, accumulate tokens until budget is exhausted."""
    kept, accumulated = [], 0
    for turn in reversed(turns):
        cost = self._estimate_turn_tokens(turn)
        if accumulated + cost > budget and kept:
            break  # Always keep at least the newest turn
        kept.append(turn)
        accumulated += cost
    kept.reverse()
    discarded = turns[:len(turns) - len(kept)]
    return kept, discarded

Then in _trim_messages(), replace Steps 2+3 with:

kept, discarded = self._token_budget_trim(turns, budget)

# Safety net: turn count cap (secondary, not primary trigger)
if len(kept) > self.max_context_turns:
    discarded = kept[:len(kept) - self.max_context_turns] + discarded
    kept = kept[-self.max_context_turns:]

if discarded and self.agent.memory_manager:
    # Reuse existing summary injection (unchanged)
    ...

What stays the same

run_stream() signature is unchanged
_truncate_historical_tool_results() (Step 0) is untouched
_build_context_summary_callback() summary injection mechanism is reused as-is
_smart_compact_to_budget() reactive overflow recovery is untouched
_estimate_message_tokens() estimation logic is untouched

Expected benefit

Scenario	Current behavior	After fix
31 short turns, 1.5K total tokens	Step 2 fires: removes 15 turns + LLM summary call	No trim (under budget)
25 heavy turns, 150K total tokens	Step 2 skips; Step 3 discards half (75K remaining, still over budget)	Discards only ~20 turns to reach 50K budget
35 turns, 200K total tokens	Step 2 removes half (17 turns); Step 3 removes more	Single pass removes exactly enough turns

I'm happy to open a PR implementing the above (≈80 lines changed) plus unit tests covering normal / under-budget / over-budget / turn-safety-net / single-turn-over-budget scenarios. Just let me know if you prefer a different approach.

中文简述

_trim_messages() 中的 Step 2（轮次数检查）在 Step 3（token 预算检查）之前执行：

31 个短轮次（每个 ~50 token，总计 ~1.5K）远低于模型上下文预算，但 Step 2 仍触发"移除一半 + LLM 摘要调用"，浪费了一次 API 调用；
两步的"移除一半"策略不感知实际需要释放多少 token——可能移除过多或不够。

建议将 Step 2 + Step 3 合并为一次 token-budget-first 扫描：从最新 turn 向前累积 token，保留不超预算的最长后缀，其余丢弃并复用现有摘要注入。max_context_turns 降级为安全上限。

改动约 80 行，不改变公共 API 签名，不引入新依赖。可直接提 PR。

[Bug] _trim_messages(): turn-count check fires before token-budget check, causing unnecessary summary LLM calls