arXiv source returns zero results for any multi-word topic (exact-phrase query on full topic string)

arXiv source returns zero results for any multi-word topic (exact-phrase query on full topic string)

Version: 3.18.4 Host: Claude Code on a headless Linux VPS CLI: arxiv-pp-cli 2026.7.2 (installed, healthy — doctor reports Config ok / Auth not required / API reachable)

Summary

arxiv.py builds its query by wrapping the entire user topic in double quotes as an exact-phrase search across all fields:

https://github.com/mvanhorn/last30days-skill/blob/main/skills/last30days/scripts/lib/arxiv.py#L70

def _build_search_query(topic: str) -> str:
    return f'all:"{_clean_phrase(topic)}"'

Topic strings are natural-language labels ("AI video generation advances"), not phrases that appear verbatim in paper titles or abstracts. So arXiv correctly returns total_results: 0, the source contributes nothing, and it silently drops out of the report's source list.

This is not a timeout or a connectivity problem. The API answers in well under a second.

Reproduction

Both commands below use the exact argument shape from _build_search_args.

The skill's actual query — 0 results:

$ time arxiv-pp-cli query --search-query 'all:"AI video generation advances"' \
    --sort-by relevance --max-results 10 --agent
1 results (live)
{
  "meta": { "source": "live" },
  "results": {
    "entries": [],
    "total_results": 0,
    "title": "arXiv Query: search_query=all:\"AI video generation advances\"&id_list=&start=0&max_results=10"
  }
}

real	0m0.584s

A shorter quoted phrase — results returned:

$ arxiv-pp-cli query --search-query 'all:"video generation"' --sort-by relevance --max-results 5 --agent
{
  "meta": { "source": "live" },
  "results": { "entries": [ { "authors": [ { "name": "Yue Ma" }, ... ] } ] }
}

Unquoted — also returns results, in 0.52s:

$ arxiv-pp-cli query --search-query 'all:video generation' --agent

Impact

• arXiv effectively never fires for realistic topics. Any topic beyond roughly two or three words is close to guaranteed zero.
• The failure is silent. arXiv simply doesn't appear in the "All agents reported back" list, so the brief looks complete while missing all academic coverage.
• This is most costly on exactly the topics where papers matter — research-heavy and technical subjects, which is the gating condition the source is documented to fire on.

Observed across two runs on the topic "AI video advances": zero paper coverage in both, on a window that included FLUX 3, Seedance 2.5, and an arXiv paper (2511.12834) that surfaced via Hacker News rather than via the arXiv source.

Suggested fixes

1. Drop the outer quotes and let arXiv do relevance matching on terms, since --sort-by relevance is already passed. Simplest change.
2. Extract key terms from the topic and quote only multi-word technical phrases (e.g. all:"video generation" AND all:diffusion).
3. Fall back on zero results — retry unquoted when the quoted query returns total_results: 0. The retry is cheap given sub-second response times.
4. Surface the empty result in the source list (e.g. arXiv: 0 items (no phrase match)) so silent absence stops reading as "nothing to report."

Note on the 30s timeouts

An earlier run on the same host reported all four arXiv attempts hitting SEARCH_TIMEOUT = 30. Those coincided with a period of heavy load on the box and I can't reproduce them now — the API consistently answers in under a second. Flagging them only for completeness; the zero-results behaviour above is separate, reproducible, and independent of load.