Add Serply as an optional keyed backend for web and Google Scholar search
Author: googioCreated Aug 25, 2026Updated Aug 25, 2026
Context
Stage 4's web augmentation (researchclaw/web/) has two search paths today:
WebSearchClient: Tavily whenTAVILY_API_KEYis set, otherwise a DuckDuckGo HTML scrape.GoogleScholarClient: thescholarlyscraper (optionalwebextra). Google blocks it fairly quickly on unattended runs, anduse_proxyonly partly helps.
There is no keyed option for Google web or Google Scholar results, and there is no way to get Scholar results at all without installing scholarly.
Proposal
Add Serply as an optional keyed backend behind the existing clients, following the same pattern Tavily uses (key from config or env, everything unchanged when the key is absent):
WebSearchClient: try Serply's/v1/search/after Tavily and before the DuckDuckGo fallback.GoogleScholarClient: query Serply's/v1/scholar/first when a key is set, fall back toscholarlyif that call fails. This also lets the Scholar client run withoutscholarlyinstalled.WebSearchConfig:serply_api_key/serply_api_key_env(defaultSERPLY_API_KEY), passed through_literature.pytoWebSearchAgentliketavily_api_key_env.- Plain
urllib, no new dependency. Tests with mocked responses for both clients plus the config parsing.
Disclosure: I work with Serply. Happy to adjust the shape (for example, a separate scholar_backend setting) if you would rather not have it inside the existing clients.
I have a branch ready and will open the PR referencing this issue.
Source: aiming-lab/AutoResearchClaw