【BUG】MaxBodySize is applied before gzip decompression
Summary
MaxBodySize is applied to the compressed response stream before Colly performs gzip decompression. As a result, a small compressed response can expand into a much larger response body after the limit has already been applied.
Affected Code
In http_backend.go, the response body is wrapped with io.LimitReader before gzip detection and decompression:
if bodySize > 0 {
bodyReader = io.LimitReader(bodyReader, int64(bodySize))
}
...
bodyReader, err = gzip.NewReader(bufReader)
...
body, err := io.ReadAll(bodyReader)This means the limit applies to compressed bytes, while io.ReadAll reads the decompressed stream.
Safe Validation Evidence
Tested locally against commit 20d31482af5f754832a753f88517f3dfa61d921f.
=== RUN TestReproH1_GzipBombUnbounded
repro_high_test.go:27: H1 = TRUE POSITIVE: 2072-byte compressed -> 2097152 bytes decompressed via io.ReadAll after LimitReader (CWE-409)
--- PASS: TestReproH1_GzipBombUnbounded (0.01s)The current master branch still contains the same code path in http_backend.go.
Expected Behavior
MaxBodySize should limit the final response body size exposed to Colly callbacks, including after decompression.
Actual Behavior
A gzip response can stay under MaxBodySize while compressed, then expand beyond that limit during decompression.
Impact
Very small compressed responses can produce much larger in-memory response bodies than the configured MaxBodySize suggests.
Suggested Fix
Apply a size limit to the decompressed stream as well, or enforce the limit while reading the final body.
Source: gocolly/colly