Keyed web backends (serper/brave/exa/parallel) return 0 results: client-side date filter drops all undated organic results

Keyed web backends (serper/brave/exa/parallel) return 0 results: client-side date filter drops all undated organic results

Summary

grounding.serper_search() almost always returns 0 items, even with a valid Serper API key that returns results. The Serper API responds 200 with organic results, but the client-side date filter discards every result whose date field is missing or unparseable — and Serper/Google rarely populates date on organic results. Net effect: the Serper web backend is effectively a no-op.

Environment

• last30days v3.18.4
• macOS, Python 3.13
• Valid SERPER_API_KEY (confirmed working via raw API call, see below)
• --web-backend serper (also selected automatically via native_web_backend)

Steps to reproduce

from datetime import date, timedelta
from scripts.lib import grounding as g

today = date.today()
dr = ((today - timedelta(days=30)).isoformat(), today.isoformat())
items, meta = g.serper_search("OpenAI", dr, "<valid_key>", count=5)
print(len(items), meta)   # -> 0  {'label': 'serper', ..., 'resultCount': 0}

Actual vs expected

• Actual: serper_search() returns [] (resultCount: 0) for common, result-rich queries.
• Expected: returns the organic results Serper actually returned.

Root cause

serper_search() sends tbs=cdr:1,cd_min:...,cd_max:..., which already constrains Google to the date window server-side. It then applies a second, client-side filter on each result's date field:

raw_date = r.get("date") or ""
pub_date = _parse_serper_date(raw_date)
if not _in_date_range(pub_date, date_range):   # <-- drops undated results
    continue

_in_date_range(None, date_range) returns False, so any result without a parseable date is dropped. In practice Serper returns date on very few organic results (often 0–1 of 5), so nearly everything is filtered out.

Evidence

Raw POST with the exact tbs the function builds:

tbs = cdr:1,cd_min:07/03/2026,cd_max:08/02/2026
organic returned by Serper: 5
  ...of those, with a `date` field: 1   (and it was "Apr 28, 2025", out of window)
_in_date_range(None, dr) => False        # the 4 undated results are all dropped
=> serper_search() returns 0

Suggested fix

The server-side cdr filter already bounds results to the window, so an unknown/unparseable date should not drop a result. Only drop results whose known date is out of range:

         raw_date = r.get("date") or ""
         pub_date = _parse_serper_date(raw_date)
-        if not _in_date_range(pub_date, date_range):
+        # `tbs=cdr:...` already constrains Google to the window server-side, and
+        # Serper rarely populates `date` on organic results. Dropping undated
+        # results filtered out ~everything (Serper returned 0 items). Only drop
+        # results whose KNOWN date is out of range.
+        if pub_date is not None and not _in_date_range(pub_date, date_range):
             continue

After this change, serper_search("OpenAI", dr, key) returns 4 results; "query x" returns 2 relevant results, etc.

Scope: all four keyed web backends are affected

This is not Serper-specific. brave_search(), exa_search(), and parallel_search() in grounding.py use the same if not _in_date_range(pub_date, date_range): continue guard on a date field that is frequently missing. So every keyed web backend (brave, exa, serper, parallel) drops undated results and can silently collapse to 0 items, leaving only the keyless floor. The fix above should be applied to all four call sites (or factored into a shared helper).