#6094·opensre

[BUG]Scheduler worker-pool exhaustion can skip tasks and starve recovery

Author: Devesh36Created Sep 8, 2026Updated Sep 9, 2026

Problem

The scheduler uses APScheduler's default worker pool. Scheduled agent tasks block while waiting for the shared turn-concurrency gate.

If enough tasks fire together, all scheduler workers can remain blocked. The recovery job uses the same executor and may not run.

APScheduler also defaults to one running instance per job with coalescing, and the scheduled jobs use a short misfire grace period. Skipped ticks do not create durable claims, so the recovery sweep cannot replay them.

Affected code

  • infrastructure/scheduling/scheduler/runner.py
  • infrastructure/scheduling/scheduler/runners.py
  • infrastructure/process/turn_capacity/slots.py

Expected behavior

Backpressure should delay work without silently losing scheduled executions or blocking the recovery mechanism.

Suggested fix

Separate scheduling from task execution using a durable queue, or configure dedicated executor capacity for recovery and task dispatch. Define an explicit policy for missed ticks: queue, coalesce, or record them durably.

Acceptance criteria

  • Recovery continues running while task execution is backlogged.
  • Simultaneous scheduled tasks do not silently disappear.
  • The missed-tick policy is documented and tested.