buzz-acp: idle-pool teardown race silently swallows a concurrently arriving message (no log trace beyond routine steer ack)
Summary
buzz-acp's idle-pool reaper (idle_pool_sleep_due, crates/buzz-acp/src/lib.rs:2164-2179, invoked from the reaper-tick select! arm at lib.rs:3606-3654) tears the agent pool back to a lazy/listening state after idle_pool_sleep_secs (900s in our config) of inactivity via shutdown_agent_pool. When a new message/wake arrives at, or just before, the same tick that triggers this teardown, the message can be silently dropped: no error, no retry record, nothing but a routine tracing::info! line —
crates/buzz-acp/src/lib.rs:3914-3922
tracing::info!(
channel = %channel_id,
event_id = %event_id,
?ack,
release_withheld,
drop_withheld,
signal_fallback,
"non-cancelling steer ack received"
);— which reads as routine steer bookkeeping, not as a dropped-message signal.
This looks like a distinct failure mode from the already-tracked #6378 / #7171. Those describe last_activity not being refreshed for genuinely in-flight work (background subagents, long turns), so the pool is torn down while the agent is still busy. Here, by contrast, the pool really is idle and due for teardown — the bug is that a message arriving concurrently with that teardown races shutdown_agent_pool / pool rebuild and is lost instead of being deferred, dispatched to the fresh pool, or re-queued.
Scope
Affects every buzz-acp managed agent, independent of subscribe mode (Mentions vs Config). Mentions-mode agents hit it far more often in practice per-wake — with fewer incidental messages to keep the pool warm between targeted wakes, the pool has usually gone fully idle and been torn down by the time the next @mention/wake lands, so the race window is entered more often relative to total wake attempts.
Evidence
The "non-cancelling steer ack received" log line is the direct fingerprint of a steer racing pool teardown/rebuild. Counts observed in our deployment's agent logs:
harness-dev-orch: 69 occurrences (highest raw count, but least visible impact — many subscribed channels mean other traffic quickly re-wakes the pool, masking the drop)stock_orch: 8 occurrencesstock-news: 5 occurrences
Real-world impact
stock-news's pre-market briefing pipeline depends on a cron-triggered T-30 wake message. On 2026-09-16, 09-17, and 09-18, this wake was accepted by the relay (accepted:true) but the agent never responded to it — 3 relay retries, then retry-exhausted — consistent with the wake landing inside an idle-pool teardown window and being swallowed.
Suggested repro / investigation
Dispatch a message exactly at/after idle_pool_sleep_bound elapses (900s default) while pool_ready is true and the idle_pool_sleep_due gates (work_queued, prompt_tasks_in_flight, wake_or_respawn_in_flight) are all clear, and trace what happens to that message through shutdown_agent_pool and pool rebuild (lib.rs:3635-3653). The race window looks like it's between the gate check passing and the teardown/rebuild actually completing, rather than in the gate logic itself — i.e. a message dispatched into the old pool concurrently with (or immediately after) shutdown_agent_pool gets a steer ack against a pool that's mid-teardown, hits one of the non-fallback SteerAck branches (lib.rs:3889-3913), and is neither retried against the new pool nor surfaced as an error.
Related issues
- #6378 — idle pool teardown kills an in-flight session, losing a queued background-task wakeup. Related mechanism (same reaper), different trigger: in-flight session activity not counted as activity, vs. a new message racing the teardown of an already-idle pool.
- #7171 — idle-pool reaper tears down right after a long turn completes. Same reaper code path, different root cause: stale
last_activitynot refreshed on turn completion, vs. a delivery race during an already-correctly-triggered teardown.
Neither of the above describes this specific scenario — a message that arrives concurrently with an already-due idle teardown being silently swallowed with no log trace beyond a routine steer-ack info line — so filing separately rather than piling onto either.
Source: block/buzz