#4375·firecrawl

Self-hosted /v2/search silently returns zero results when a DuckDuckGo result URL has a malformed percent escape

Author: tkaufmannCreated Aug 20, 2026Updated Sep 24, 2026

What happened

On a self-hosted instance without Fire-engine and without SEARXNG_ENDPOINT, POST /v2/search answers 200 with

json
{"success": true, "data": {"web": []}}

for some queries, while the API log shows the search actually crashed:

error [api/v2:searchController]: Error in search function
  {"query":"python asyncio tutorial","error":{"name":"URIError","message":"URI malformed",
   "stack":"URIError: URI malformed\n    at decodeURIComponent (<anonymous>)
   \n    at cleanUrl (/app/dist/src/search/v2/ddgsearch.js:60:23)
   \n    at extractResults (/app/dist/src/search/v2/ddgsearch.js:80:25)
   \n    at ddgSearch (/app/dist/src/search/v2/ddgsearch.js:168:36)
   \n    at async search (/app/dist/src/search/v2/index.js:38:28)
   \n    at async executeSearch (/app/dist/src/search/execute.js:19:29)
   \n    at async searchController (/app/dist/src/controllers/v2/search.js:118:24)"}}

The caller cannot tell "nothing found" from "the search blew up", because the response says success: true either way. That is the part that hurts: a broken search is indistinguishable from an honest empty result.

Where it comes from

apps/api/src/search/v2/ddgsearch.ts:

typescript
function cleanUrl(href: string): string {
  if (href.includes("uddg=")) {
    const url = new URL(href, "https://duckduckgo.com");
    const uddg = url.searchParams.get("uddg");
    return uddg ? decodeURIComponent(uddg) : href;   // line 37
  }
  return href;
}

decodeURIComponent throws URIError on a malformed percent escape. It is called per result inside the extraction loop (line 67), so one bad result URL aborts the extraction of the whole page, and with it the entire search.

How to reproduce

It depends on what DuckDuckGo returns for a given query, not on the query encoding. Measured on 2026-08-21 against ghcr.io/firecrawl/firecrawl:2.10.19, self-hosted with docker compose, USE_DB_AUTHENTICATION=false:

Query Result
python asyncio tutorial web: [] + URIError in the log
Hüttenberg Hessen Rathaus web: [] + URIError in the log
Gießen Stadtverwaltung 3 results
firecrawl open source 3 results

So it is not reproducible from the query alone — it depends on the result set of the moment.

Suggested fix

Two small things, independent of each other:

  1. Make cleanUrl survive a single bad URL:
typescript
function cleanUrl(href: string): string {
  if (href.includes("uddg=")) {
    const url = new URL(href, "https://duckduckgo.com");
    const uddg = url.searchParams.get("uddg");
    if (!uddg) return href;
    try {
      return decodeURIComponent(uddg);
    } catch {
      return uddg;   // or `href`, or skip this result
    }
  }
  return href;
}
  1. Do not report success: true when the search backend threw. An empty list after a crash is a silent failure; the caller has no way to retry or warn.

Workaround

Pointing SEARXNG_ENDPOINT at a local SearXNG avoids the code path entirely, since search() only falls through to DuckDuckGo when SearXNG returns nothing.