#24051·loki

indexgateway: Cap client-side in-flight requests to avoid retry amplification when all replicas shed

Author: trevorwhitneyCreated Aug 18, 2026Updated Sep 14, 2026
Labelscomponent/index-gateway

With -index-gateway.max-concurrent enabled, GatewayClient.poolDo retries a shed request against the next replica, which is the behavior we want: another replica may be less loaded.

When every replica is saturated the same mechanism amplifies load. poolDo passes maxRetries = -1 for six of the seven RPCs, so it walks the entire address set, and in simple mode that set is every discovered gateway. A single request can therefore wait up to max-concurrent-queue-timeout per replica before failing, and each of those waits occupies a queue slot on a gateway that is already overloaded.

A client-side cap on in-flight index gateway requests would let the client fail fast once it can see the whole pool is shedding, instead of serially probing every replica.

Follow-up to https://github.com/grafana/loki/pull/23932.