[Bug]: Brave web search forwards the raw HTML challenge page as the error message

Author: HuntercodeTCreated Sep 16, 2026Updated Sep 16, 2026

Summary

Both Brave web-search error paths in lib/web-search/brave.ts forward the upstream response body verbatim:

typescript

if (!res.ok) {
  const errorText = await res.text().catch(() => '');
  throw new Error(`Brave Search error (${res.status}): ${errorText || res.statusText}`);
}

When Brave throttles or blocks a request it does not answer with a short JSON error — it answers with a full HTML challenge/error page. That page becomes the Error.message, propagates up through searchWithBrave, and ends up in logs and in the classroom generation UI as a wall of markup. A rate-limit, which is transient and actionable, is also indistinguishable from a real outage.

This is the keyless scrape path most users hit, since Brave is requiresApiKey: false (see #641).

Steps to Reproduce

Configure Brave as the web search provider without an API key (keyless scrape mode).
Issue enough searches to trip Brave's public-endpoint rate limit, or mock proxyFetch to resolve with an HTML body and status 429.
Observe the thrown error.

Current Behaviour

Error.message contains the entire HTML document, e.g. Brave Search error (429): <!doctype html><html>…. The message is unbounded in length and carries no usable signal.

Expected Behaviour

429 produces a dedicated, actionable message that names the throttling and the available remedies.
Non-429 failures fall back to the HTTP status text rather than echoing a markup body.
Short plain-text upstream details are still preserved, since those are genuinely useful.

#643 — same class of defect for TTS providers, where a 429 was collapsed into a generic 500.
#641, #687 — prior fixes to the same keyless Brave path.

Environment

Affected file: lib/web-search/brave.ts (both searchWithBraveApi and searchWithBraveScrape)
Reproduced against main @ 2cbd011c

I have a fix ready and will open a PR against this issue.

Source: THU-MAIC/OpenMAIC