#23754·loki

Thanos Object Store Client hammers s3 If http 429 SlowDown received

Author: NachtfalkeawCreated Aug 4, 2026Updated Sep 12, 2026
Labelsstorage/thanos

Describe the bug If the s3 storage sends http 429 with "SlowDown" and "Reduced your rewuest rate" the thanos Client continues to Hammer the s3 leading to more 429 SlowDown which makes the Situation worse than better.

To Reproduce Steps to reproduce the behavior:

  1. Started Loki 3.7.3 monolithic
  2. Created many requests to s3 and Run into rate limit

Expected behavior The thanos Object Store Client should use a backoff mechanism to Slow down and wir If Errors Like 429 SlowDown are received to give the s3 the Chance to unblock the Client.

Further our local Nutanix s3 May send Errors Like http 503 "StorageFull" and s3 still Hammers the s3. It should wait and slow down to Reduce Error rate.

Environment:

  • Infrastructure: binary, baremetal
  • Deployment tool: binary, monolithic

Screenshots, Alloy config, or terminal output N/A