Feature request: anybrowse MCP integration for Cloudflare-protected scraping

Author: kc23goCreated Mar 21, 2026Updated Sep 8, 2026
Labelsbugconfirmed

Problem

GPT-Researcher's web retrieval fails silently on Cloudflare-protected sites (major news outlets, LinkedIn, Amazon, government sites). The scraper gets a 403 or returns empty content without the research loop knowing.

Proposed integration

anybrowse is an MCP-native web scraper with Cloudflare bypass via residential Chrome. It fits naturally into the GPT-Researcher retrieval chain as a fallback scraper when standard HTTP fails.

It exposes a scrape tool that returns clean LLM-ready markdown -- the exact format GPT-Researcher needs for its context window.

Quick test

Add as an MCP server:

json
{
  "mcpServers": {
    "anybrowse": {
      "type": "streamable-http",
      "url": "https://anybrowse.dev/mcp"
    }
  }
}

Then: "Research the latest AI regulation news from EU government sites" -- anybrowse handles the Cloudflare-protected .europa.eu domains that standard scrapers bounce off.

Details

  • Free tier: 10 scrapes/day, no API key
  • Paid: $5 for 3,000 credits (never expire)
  • MCP tools: scrape, crawl, batch_scrape, extract, search
  • Docs: https://anybrowse.dev/docs

Would be happy to contribute a retriever class if this seems like a good fit for the project.

Source: assafelovic/gpt-researcher