Passing an empty visited_urls set breaks shared URL-dedup state

Summary

GPTResearcher.__init__() currently initializes visited_urls with:

self.visited_urls = visited_urls or set()

That replaces a caller-provided empty set with a new set because an empty set is falsy.

This matters because visited_urls is intentionally used as shared mutable state between parent/sub-researchers so already-scraped URLs can be deduplicated across related research runs. The researcher code explicitly documents that visited_urls may be shared with a parent researcher and is deliberately not cleared.

Minimal reproduction

Conceptually:

shared = set()
researcher = GPTResearcher(query="test", visited_urls=shared)

assert researcher.visited_urls is shared  # currently fails when shared is empty

If shared already contains at least one URL, the identity is preserved. The bug therefore only appears at the most common initial state: an empty shared set.

Why this matters

Parent/sub-researcher flows can pass an empty accumulator expecting subsequent discoveries to be visible through the same set object. Because the constructor swaps that object out on first use, the parent does not observe the child’s first URL updates through shared-state identity.

That weakens cross-research URL deduplication and can lead to duplicate scraping/retrieval work, especially in deep/detailed research flows that create nested GPTResearcher instances.

Expected behavior

Only None should allocate a new set. A caller-supplied set should be preserved even when empty:

self.visited_urls = visited_urls if visited_urls is not None else set()

Regression coverage

A focused test should verify both cases:

shared = set()
researcher = GPTResearcher(..., visited_urls=shared)
assert researcher.visited_urls is shared

prepopulated = {"https://example.com"}
researcher = GPTResearcher(..., visited_urls=prepopulated)
assert researcher.visited_urls is prepopulated

The first assertion is the regression case; the second locks in current non-empty behavior.

I searched the issue tracker for visited_urls, empty-set identity, and shared parent/sub-researcher URL state and did not find an existing report for this constructor bug.

AI-assisted review disclosure: I used an AI coding assistant to inspect the constructor and shared-state call paths and to help draft this report. The issue is based on current source behavior; no benchmark or runtime performance claim is being made.