ffuf fails to match / filter encoded content due Accept-Encoding header
Author: SanderWindCreated Mar 1, 2022Updated Feb 1, 2026
Hi Joona!
A few weeks ago I noticed that ffuf didn't match my provided regular expression filters / matchers. Today, I encountered this again and did a bit more research on how and why. It looks like when ffuf is configured with a request using the Accept-Encoding header, the response is not decoded before the matchers / filters check the response.
Reproducing the problem
I did a simple test with a single-line wordlist, containing test. The URL I'm checking contains the following:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>bandjes</title>
</head>
<body>
<h1>Hello</h1>
<p>I'm a bug bounty hunter and security researcher.</p>
<p>Check my profile on intigriti: <a href="https://app.intigriti.com/profile/bandjes" rel="noopener">https://app.intigriti.com/profile/bandjes</a></p>
</body>
</html>Without Accept-Encoding
$ ffuf -u https://www.bandj.es/\?FUZZ -mr 'Hello' -w ~/Bug\ Bounty/00_tools/wordlists/single.txt
/'___\ /'___\ /'___\
/\ \__/ /\ \__/ __ __ /\ \__/
\ \ ,__\\ \ ,__\/\ \/\ \ \ \ ,__\
\ \ \_/ \ \ \_/\ \ \_\ \ \ \ \_/
\ \_\ \ \_\ \ \____/ \ \_\
\/_/ \/_/ \/___/ \/_/
v1.3.1-dev
________________________________________________
:: Method : GET
:: URL : https://www.bandj.es/?FUZZ
:: Wordlist : FUZZ: /Users/sander/Bug Bounty/00_tools/wordlists/single.txt
:: Follow redirects : false
:: Calibration : false
:: Timeout : 10
:: Threads : 40
:: Matcher : Regexp: Hello
________________________________________________
test [Status: 200, Size: 876, Words: 189, Lines: 29]
:: Progress: [1/1] :: Job [1/1] :: 0 req/sec :: Duration: [0:00:00] :: Errors: 0 ::With Accept-Encoding
Now executing ffuf again with Accept-Encoding: gzip.
$ ffuf -u https://www.bandj.es/\?FUZZ -H 'Accept-Encoding: gzip' -mr 'Hello' -w ~/Bug\ Bounty/00_tools/wordlists/single.txt
/'___\ /'___\ /'___\
/\ \__/ /\ \__/ __ __ /\ \__/
\ \ ,__\\ \ ,__\/\ \/\ \ \ \ ,__\
\ \ \_/ \ \ \_/\ \ \_\ \ \ \ \_/
\ \_\ \ \_\ \ \____/ \ \_\
\/_/ \/_/ \/___/ \/_/
v1.3.1-dev
________________________________________________
:: Method : GET
:: URL : https://www.bandj.es/?FUZZ
:: Wordlist : FUZZ: /Users/sander/Bug Bounty/00_tools/wordlists/single.txt
:: Header : Accept-Encoding: gzip
:: Follow redirects : false
:: Calibration : false
:: Timeout : 10
:: Threads : 40
:: Matcher : Regexp: Hello
________________________________________________
:: Progress: [1/1] :: Job [1/1] :: 0 req/sec :: Duration: [0:00:00] :: Errors: 0 ::Response logging
To check the response of the requests, I performed above commands again but now without the regular expression match.
Without Accept-Encoding
$ ffuf -u https://www.bandj.es/\?FUZZ -w ~/Bug\ Bounty/00_tools/wordlists/single.txt -od /tmp/ffuf-test -of md
/'___\ /'___\ /'___\
/\ \__/ /\ \__/ __ __ /\ \__/
\ \ ,__\\ \ ,__\/\ \/\ \ \ \ ,__\
\ \ \_/ \ \ \_/\ \ \_\ \ \ \ \_/
\ \_\ \ \_\ \ \____/ \ \_\
\/_/ \/_/ \/___/ \/_/
v1.3.1-dev
________________________________________________
:: Method : GET
:: URL : https://www.bandj.es/?FUZZ
:: Wordlist : FUZZ: /Users/sander/Bug Bounty/00_tools/wordlists/single.txt
:: Follow redirects : false
:: Calibration : false
:: Timeout : 10
:: Threads : 40
:: Matcher : Response status: 200,204,301,302,307,401,403,405
________________________________________________
[Status: 200, Size: 497, Words: 55, Lines: 16]
| RES | 97ea09a64418a648ca12cd0d7d9f8537
* FUZZ: test
:: Progress: [1/1] :: Job [1/1] :: 0 req/sec :: Duration: [0:00:00] :: Errors: 0 ::$ cat /tmp/ffuf-test/97ea09a64418a648ca12cd0d7d9f8537
───────┬─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
│ File: /tmp/ffuf-test/97ea09a64418a648ca12cd0d7d9f8537
───────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
1 │ GET /?test HTTP/1.1
2 │ Host: www.bandj.es
3 │ User-Agent: Fuzz Faster U Fool v1.3.1-dev
4 │ Accept-Encoding: gzip
5 │
6 │
7 │ ---- ↑ Request ---- Response ↓ ----
8 │
9 │ HTTP/1.1 200 OK
10 │ Transfer-Encoding: chunked
11 │ Connection: keep-alive
12 │ Content-Type: text/html; charset=utf-8
13 │ Date: Tue, 01 Mar 2022 10:03:22 GMT
14 │ Etag: W/"621ded6c-1f1"
15 │ Last-Modified: Tue, 01 Mar 2022 09:54:52 GMT
16 │ Server: nginx/1.14.0 (Ubuntu)
17 │ X-Content-Type-Options: nosniff
18 │ X-Frame-Options: SAMEORIGIN
19 │ X-Xss-Protection: 1; mode=block
20 │
21 │ 1f1
22 │ <!DOCTYPE html>
23 │ <html lang="en">
24 │ <head>
25 │ <meta charset="UTF-8">
26 │ <meta http-equiv="X-UA-Compatible" content="IE=edge">
27 │ <meta name="viewport" content="width=device-width, initial-scale=1.0">
28 │ <title>bandjes</title>
29 │ </head>
30 │ <body>
31 │ <h1>Hello</h1>
32 │
33 │ <p>I'm a bug bounty hunter and security researcher.</p>
34 │ <p>Check my profile on intigriti: <a href="https://app.intigriti.com/profile/bandjes" rel="noopener">https://app.intigriti.com/profile/bandjes</
│ a></p>
35 │ </body>
36 │ </html>
37 │
38 │ 0
39 │
───────┴─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────With Accept-Encoding
$ ffuf -u https://www.bandj.es/\?FUZZ -H 'Accept-Encoding: gzip' -w ~/Bug\ Bounty/00_tools/wordlists/single.txt -od /tmp/ffuf-test -of md
/'___\ /'___\ /'___\
/\ \__/ /\ \__/ __ __ /\ \__/
\ \ ,__\\ \ ,__\/\ \/\ \ \ \ ,__\
\ \ \_/ \ \ \_/\ \ \_\ \ \ \ \_/
\ \_\ \ \_\ \ \____/ \ \_\
\/_/ \/_/ \/___/ \/_/
v1.3.1-dev
________________________________________________
:: Method : GET
:: URL : https://www.bandj.es/?FUZZ
:: Wordlist : FUZZ: /Users/sander/Bug Bounty/00_tools/wordlists/single.txt
:: Header : Accept-Encoding: gzip
:: Follow redirects : false
:: Calibration : false
:: Timeout : 10
:: Threads : 40
:: Matcher : Response status: 200,204,301,302,307,401,403,405
________________________________________________
[Status: 200, Size: 317, Words: 2, Lines: 2]
| RES | ca2ef9213b3ef1f284f07d7ed327d362
* FUZZ: test
:: Progress: [1/1] :: Job [1/1] :: 0 req/sec :: Duration: [0:00:00] :: Errors: 0 ::$ cat /tmp/ffuf-test/ca2ef9213b3ef1f284f07d7ed327d362
───────┬─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
│ File: /tmp/ffuf-test/ca2ef9213b3ef1f284f07d7ed327d362
───────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
1 │ GET /?test HTTP/1.1
2 │ Host: www.bandj.es
3 │ User-Agent: Fuzz Faster U Fool v1.3.1-dev
4 │ Accept-Encoding: gzip
5 │
6 │
7 │ ---- ↑ Request ---- Response ↓ ----
8 │
9 │ HTTP/1.1 200 OK
10 │ Transfer-Encoding: chunked
11 │ Connection: keep-alive
12 │ Content-Encoding: gzip
13 │ Content-Type: text/html; charset=utf-8
14 │ Date: Tue, 01 Mar 2022 10:00:01 GMT
15 │ Etag: W/"621ded6c-1f1"
16 │ Last-Modified: Tue, 01 Mar 2022 09:54:52 GMT
17 │ Server: nginx/1.14.0 (Ubuntu)
18 │ X-Content-Type-Options: nosniff
19 │ X-Frame-Options: SAMEORIGIN
20 │ X-Xss-Protection: 1; mode=block
21 │
22 │ 13d
23 │ ^_^@^@^@^@^@^@^C�QMO^C!^T��W<�xq^W{3^M��ZcOzh^S=����%˶鿗v��^@yo�^L3��^^�����^U�쬘��^AV��^S���@ي^Y��^\f J��c�d�y����/�s�^U~^Nf��k�����E�Mc��
24 │ >�/���c��^E�K���^M^^bH�����Y�^V�Fau.n�x���U��E>�o����^VE#}��=�c9ct����^^'�z.���P���^ZcD��v �^Y:h���^Qt�1A^Q�^^ՐL�%�Q&�1Ռ�I3��F�^A1���^W��t�d^V�$�;NN��/(�1�?x���^S�N^YH��r�C��1^Q�o&
│ �R��1:�.I�^_�^E�#^R��^A^@^@
25 │ 0
26 │
───────┴─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────As shown in the latest request response it's all gibberish and feels like ffuf doesn't decode the content before performing its matchers / filters.
Normally, it would be possible to leave the Accept-Encoding header out of the request, but sometimes, Cloudflare does require this header to be present or it will block your request.
Version
I'm using ffuf version 1.3.1-dev.
Source: ffuf/ffuf