Prefetched sources (requires_scraping=False) never enter visited_urls, so they don't dedupe and drop out of References
_search_relevant_source_urls splits retriever results into new_search_urls (scraped, deduped via _get_new_urls against visited_urls) and prefetched_content (retrievers that declare requires_scraping=False, e.g. PubMed Central, or trip the legacy long-raw_content heuristic). The prefetched branch calls add_research_sources() but never touches visited_urls.
Two effects follow. A prefetched source hit by two different sub-queries gets merged into scraped_content twice instead of deduping the way a scraped URL would through _get_new_urls. And add_references() builds the report's References section purely from visited_urls, so the source's own citation never shows up even though its text is in the report.
Reproduced against the real ResearchConductor._search_relevant_source_urls and add_references at HEAD 6f99857: a retriever declaring requires_scraping=False returns the same URL across two sub-queries. research_sources grows to 2 entries for it, visited_urls stays set(), and the generated References section comes back empty.
Fix looks small: add the URL to visited_urls in the requires_scraping is False and legacy-heuristic branches too (researcher.py around lines 876-888), the same way _get_new_urls already does for scraped URLs. Happy to send a PR if that's useful.
Source: assafelovic/gpt-researcher