[BUG] Kompress retry overflow causes /v1/compress to return 503
We're seeing /v1/compress return 503 after repeated background Kompress loading failures on Headroom 0.35.0, Python 3.13, Linux/Kubernetes.
The service traceback points to:
kompress_compressor.py:1089, _download_retry_blocked
_DOWNLOAD_RETRY_BASE_SECONDS * (2 ** (failures - 1))
OverflowError: int too large to convert to float
The delay is meant to stop growing at 300 seconds, but the exponential calculation happens before min() applies that cap. It overflows when the failure count reaches 1,025.
Minimal reproduction
for failures in (1024, 1025):
try:
print(failures, min(300.0, 5.0 * (2 ** (failures - 1))))
except OverflowError as exc:
print(failures, type(exc).__name__, str(exc))
1024 300.0
1025 OverflowError int too large to convert to float
In our setup, offline mode is enabled and the ModernBERT tokenizer is missing from the cache. That is a separate deployment problem, but repeated loading failures should keep a capped retry delay rather than crash compression. Other compression paths continued working, so this only affected some requests.
Could the backoff be capped before calculating the exponent? Affected code.
Source: headroomlabs-ai/headroom