[Good First Issue] Request fingerprint changes based on header order
Author: yetvalCreated Sep 28, 2026Updated Sep 28, 2026
What would you like to share?
When fp_include_headers=True, two requests with the same headers can get different fingerprints if the headers are added in a different order.
As shown in the Example:
from scrapling.spiders import Request
r1 = Request("https://example.com", headers={"Accept": "text/html", "User-Agent": "x"})
r2 = Request("https://example.com", headers={"User-Agent": "x", "Accept": "text/html"})
print(r1.update_fingerprint(include_headers=True) == r2.update_fingerprint(include_headers=True))Actual: False
Expected:True
This can cause the spider to treat identical requests as different and fetch the same page more than once. A solution to fix this issue should make header order not affect the request fingerprint and include a regression test for this behavior.
Additional information
No response
Source: D4Vinci/Scrapling