#923·colly

【BUG】MaxBodySize is applied before gzip decompression

Author: GG-FengCreated Jul 31, 2026Updated Aug 14, 2026

Summary

MaxBodySize is applied to the compressed response stream before Colly performs gzip decompression. As a result, a small compressed response can expand into a much larger response body after the limit has already been applied.

Affected Code

In http_backend.go, the response body is wrapped with io.LimitReader before gzip detection and decompression:

go
if bodySize > 0 {
	bodyReader = io.LimitReader(bodyReader, int64(bodySize))
}
...
bodyReader, err = gzip.NewReader(bufReader)
...
body, err := io.ReadAll(bodyReader)

This means the limit applies to compressed bytes, while io.ReadAll reads the decompressed stream.

Safe Validation Evidence

Tested locally against commit 20d31482af5f754832a753f88517f3dfa61d921f.

=== RUN   TestReproH1_GzipBombUnbounded
    repro_high_test.go:27: H1 = TRUE POSITIVE: 2072-byte compressed -> 2097152 bytes decompressed via io.ReadAll after LimitReader (CWE-409)
--- PASS: TestReproH1_GzipBombUnbounded (0.01s)

The current master branch still contains the same code path in http_backend.go.

Expected Behavior

MaxBodySize should limit the final response body size exposed to Colly callbacks, including after decompression.

Actual Behavior

A gzip response can stay under MaxBodySize while compressed, then expand beyond that limit during decompression.

Impact

Very small compressed responses can produce much larger in-memory response bodies than the configured MaxBodySize suggests.

Suggested Fix

Apply a size limit to the decompressed stream as well, or enforce the limit while reading the final body.