`AutoscaledPool` controller does not converge to optimal `desired_concurency`
AutoscaledPool controller does not converge to optimal desired_concurency which can lead to steady decline in performance for long running actors. It seems that current version of AutoscaledPool controller can keep increasing desired_concurrency far beyond the optimal level and thus gradually decrease the Actor performance. The controlled value of desired_concurrency does not even converge to the optimal value.
Experiment with context:
One Actor is started twice with 2 different settings. One limits the desired concurrency to 10 other keeps it unlimited. The Actor is running code that is more or less constant load (within the possibilities of the test). It is expected that AutoscaledPool will converge to the optimal value of desired_concurrency with such constant load. That is not the case and the Actor with unbounded desired_concurency will keep increasing it while steadily dropping in performance. The Actor with bounded desired_concurency keeps constant performance, which is far better. This means that the controller of the AutoscaledPool fails to find the optimal value, overshoots it and continues to diverge from the optimal value.
Example numbers: Actor with fixed concurrency: Runtime 60, desired_concurency = 10 , requests_finished_per_minute = 2012 Runtime 1260s, desired_concurency = 10 , requests_finished_per_minute = 2126
Actor with unbounded desired concurrency: Runtime 60, desired_concurency = 15 , requests_finished_per_minute = 1930 Runtime 1680s, desired_concurency = 52 , requests_finished_per_minute = 1166
Source: apify/crawlee-python