Content-Encoding with multiple codings is not decoded
Describe the bug
RFC 9110 §8.4 allows Content-Encoding to list several codings. The client must decode them in reverse order. When a server sends a body compressed twice and the header Content-Encoding: gzip,gzip, aiohttp does not decode the body. It passes the raw gzip bytes to resp.json() (and resp.text()), which then tries to UTF-8-decode the gzip magic and fails.
To Reproduce
Two files. No TLS, no proxy.
server.py (stdlib only) always answers with a doubly-gzipped JSON body and declares Content-Encoding: gzip,gzip:
#!/usr/bin/env python3
import gzip
import json
import socket
body = gzip.compress(gzip.compress(json.dumps({"hello": "world"}).encode()))
srv = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
srv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
srv.bind(("127.0.0.1", 8080))
srv.listen()
while True:
conn, _ = srv.accept()
conn.recv(65536)
head = "\r\n".join((
"HTTP/1.1 200 OK",
"Content-Type: application/json",
"Content-Encoding: gzip,gzip",
f"Content-Length: {len(body)}",
"Connection: close",
"",
"",
)).encode()
conn.sendall(head + body)
conn.close()client.py:
#!/usr/bin/env python3
import asyncio
import aiohttp
async def fetch():
async with aiohttp.ClientSession() as session:
async with session.get("http://127.0.0.1:8080/") as resp:
print(await resp.json())
asyncio.run(fetch())Run:
./server.py &
uv run --with aiohttp==3.14.3 ./client.pyExpected behavior
aiohttp decodes each coding listed in Content-Encoding, in reverse order (here: gunzip twice), and resp.json() returns {"hello": "world"}.
Logs/tracebacks
$ ./server.py &
$ uv run --with aiohttp==3.14.3 ./client.py
Installed 9 packages in 8ms
Traceback (most recent call last):
File "/tmp/tg/./client.py", line 12, in <module>
asyncio.run(fetch())
~~~~~~~~~~~^^^^^^^^^
File "/home/philip/.local/share/uv/python/cpython-3.13.11-linux-x86_64-gnu/lib/python3.13/asyncio/runners.py", line 195, in run
return runner.run(main)
~~~~~~~~~~^^^^^^
File "/home/philip/.local/share/uv/python/cpython-3.13.11-linux-x86_64-gnu/lib/python3.13/asyncio/runners.py", line 118, in run
return self._loop.run_until_complete(task)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
File "/home/philip/.local/share/uv/python/cpython-3.13.11-linux-x86_64-gnu/lib/python3.13/asyncio/base_events.py", line 725, in run_until_complete
return future.result()
~~~~~~~~~~~~~^^
File "/tmp/tg/./client.py", line 9, in fetch
print(await resp.json())
^^^^^^^^^^^^^^^^^
File "/home/philip/.cache/uv/archive-v0/6kdapgyXRUIN9ElB/lib/python3.13/site-packages/aiohttp/client_reqrep.py", line 795, in json
return loads(stripped.decode(encoding))
~~~~~~~~~~~~~~~^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start bytePython Version
$ uv run --with aiohttp==3.14.3 python --version
Python 3.13.11aiohttp Version
$ uv run --with aiohttp==3.14.3 python -m pip show aiohttp
Name: aiohttp
Version: 3.14.3
Summary: Async http client/server framework (asyncio)
Home-page: https://github.com/aio-libs/aiohttp
Author:
Author-email:
License: Apache-2.0 AND MIT
Location: /home/philip/.cache/uv/archive-v0/6kdapgyXRUIN9ElB/lib/python3.13/site-packages
Requires: aiohappyeyeballs, aiosignal, attrs, frozenlist, multidict, propcache, yarl
Required-by:multidict Version
$ uv run --with aiohttp==3.14.3 python -m pip show multidict
Name: multidict
Version: 6.7.1
Summary: multidict implementation
Home-page: https://github.com/aio-libs/multidict
Author: Andrew Svetlov
Author-email: [email protected]
License: Apache License 2.0
Location: /home/philip/.cache/uv/archive-v0/6kdapgyXRUIN9ElB/lib/python3.13/site-packages
Requires:
Required-by: aiohttp, yarlpropcache Version
$ uv run --with aiohttp==3.14.3 python -m pip show propcache
Name: propcache
Version: 0.5.2
Summary: Accelerated property cache
Home-page: https://github.com/aio-libs/propcache
Author: Andrew Svetlov
Author-email: [email protected]
License: Apache-2.0
Location: /home/philip/.cache/uv/archive-v0/6kdapgyXRUIN9ElB/lib/python3.13/site-packages
Requires:
Required-by: aiohttp, yarlyarl Version
$ uv run --with aiohttp==3.14.3 python -m pip show yarl
Name: yarl
Version: 1.24.5
Summary: Yet another URL library
Home-page: https://github.com/aio-libs/yarl
Author: Andrew Svetlov
Author-email: [email protected]
License: Apache-2.0
Location: /home/philip/.cache/uv/archive-v0/6kdapgyXRUIN9ElB/lib/python3.13/site-packages
Requires: idna, multidict, propcache
Required-by: aiohttpOS
Arch Linux
Related component
Client
Additional context
Suspected cause: HttpParser.parse_headers() in aiohttp/http_parser.py:632-635 matches the whole header against the set {"gzip", "deflate", "br", "zstd"}. A value like gzip,gzip does not match, so encoding stays None (line 608), the DeflateBuffer wrapper is
skipped (line 882), and raw bytes pass through. Multi-coding Content-Encoding is simply not implemented.
Code of Conduct
- I agree to follow the aio-libs Code of Conduct
Source: aio-libs/aiohttp