[Bug]: CancelDispatcher stops listening after a Pub/Sub disconnect
Prerequisites
- I searched existing issues and PRs; this is not a duplicate.
- This is a bug, not a usage question.
Background / Description
CancelDispatcher owns three long-lived subscriptions: session cancel, background-task cancel, and session interrupt. Each loop wraps a single async for message_bus.subscribe(...) in try/except Exception, logs the error, and returns. RedisMessageBus.subscribe() propagates connection errors other than its handled idle TimeoutError. A transient Pub/Sub disconnect therefore permanently stops the affected cancel loop until the whole app process restarts. __aenter__() can still return because each loop sets its ready event in finally, so the degraded process looks started while later cancellation signals go unhandled.
Observed on main at d0f4be4ae8cf3a111b5435466d0bee2808ee706a. This affects cross-process cancellation and interruption of in-flight Agent runs and background tools.
Error Messages
ERROR CancelDispatcher session-cancel loop crashed.
ConnectionError: simulated Redis disconnect
subscription attempts: 1; session loop done: True; in-flight chat run cancelled: FalseSteps to Reproduce
- Use a
MessageBustest double whosesubscribe(session_cancel_channel, on_ready=...)callson_ready()and then raisesConnectionErroronce; subsequent subscriptions would yield normally. - Enter
CancelDispatcher, register an in-flight chat task inChatRunRegistry, and let the first subscription fail. - Publish a session-cancel payload after the fake bus is available again.
- Observe that
CancelDispatcher._session_task.done()isTrue, subscription was attempted only once, and the chat task keeps running. The task-cancel and interrupt loops have the same one-shot structure.
The fault injection requires no Redis service; it reproduces the failure at the MessageBus.subscribe() boundary.
Expected behavior / Proposed fix
Each cancel subscription should resubscribe after a transient error or unexpected normal termination, with bounded backoff and a traceback-bearing log. The loops should remain independently recoverable, and shutdown should cancel and await any pending retry. A regression test should prove a post-reconnect signal cancels its intended local task and that all three channels reconnect.
Pub/Sub cannot replay broadcasts published while disconnected. This proposal restores handling of subsequent signals; durable delivery during the outage would require a separate design.
Environment
- AgentScope: 2.0
main@ d0f4be4ae8cf3a111b5435466d0bee2808ee706a - Python: 3.11
- OS: Windows 11 (logic is platform-independent)
Source: agentscope-ai/agentscope