Self-hosted /v2/search silently returns zero results when a DuckDuckGo result URL has a malformed percent escape
What happened
On a self-hosted instance without Fire-engine and without SEARXNG_ENDPOINT,
POST /v2/search answers 200 with
{"success": true, "data": {"web": []}}for some queries, while the API log shows the search actually crashed:
error [api/v2:searchController]: Error in search function
{"query":"python asyncio tutorial","error":{"name":"URIError","message":"URI malformed",
"stack":"URIError: URI malformed\n at decodeURIComponent (<anonymous>)
\n at cleanUrl (/app/dist/src/search/v2/ddgsearch.js:60:23)
\n at extractResults (/app/dist/src/search/v2/ddgsearch.js:80:25)
\n at ddgSearch (/app/dist/src/search/v2/ddgsearch.js:168:36)
\n at async search (/app/dist/src/search/v2/index.js:38:28)
\n at async executeSearch (/app/dist/src/search/execute.js:19:29)
\n at async searchController (/app/dist/src/controllers/v2/search.js:118:24)"}}The caller cannot tell "nothing found" from "the search blew up", because the
response says success: true either way. That is the part that hurts: a broken
search is indistinguishable from an honest empty result.
Where it comes from
apps/api/src/search/v2/ddgsearch.ts:
function cleanUrl(href: string): string {
if (href.includes("uddg=")) {
const url = new URL(href, "https://duckduckgo.com");
const uddg = url.searchParams.get("uddg");
return uddg ? decodeURIComponent(uddg) : href; // line 37
}
return href;
}decodeURIComponent throws URIError on a malformed percent escape. It is
called per result inside the extraction loop (line 67), so one bad result
URL aborts the extraction of the whole page, and with it the entire search.
How to reproduce
It depends on what DuckDuckGo returns for a given query, not on the query
encoding. Measured on 2026-08-21 against
ghcr.io/firecrawl/firecrawl:2.10.19, self-hosted with docker compose,
USE_DB_AUTHENTICATION=false:
| Query | Result |
|---|---|
python asyncio tutorial |
web: [] + URIError in the log |
Hüttenberg Hessen Rathaus |
web: [] + URIError in the log |
Gießen Stadtverwaltung |
3 results |
firecrawl open source |
3 results |
So it is not reproducible from the query alone — it depends on the result set of the moment.
Suggested fix
Two small things, independent of each other:
- Make
cleanUrlsurvive a single bad URL:
function cleanUrl(href: string): string {
if (href.includes("uddg=")) {
const url = new URL(href, "https://duckduckgo.com");
const uddg = url.searchParams.get("uddg");
if (!uddg) return href;
try {
return decodeURIComponent(uddg);
} catch {
return uddg; // or `href`, or skip this result
}
}
return href;
}- Do not report
success: truewhen the search backend threw. An empty list after a crash is a silent failure; the caller has no way to retry or warn.
Workaround
Pointing SEARXNG_ENDPOINT at a local SearXNG avoids the code path entirely,
since search() only falls through to DuckDuckGo when SearXNG returns nothing.
Source: firecrawl/firecrawl