Share one concurrency budget between multiple crawlers

Author: vdusekCreated Aug 7, 2026Updated Aug 7, 2026
Labelsenhancementt-tooling

Each crawler builds its own AutoscaledPool and SystemStatus, so several crawlers in one process oversubscribe the host - there is no way to cap their combined concurrency. JS v4 split the scaling logic out into a ConcurrencySystem that can be constructed once and passed to several crawlers.

JS implementation: apify/crawlee#3917

✍️ Drafted by Claude Code