bug: --output with a {?}-wildcard username silently overwrites the results file (data loss)
Summary
--output FILE is guarded to single-username use, but the guard counts raw CLI args (len(args.username) at sherlock_project/sherlock.py:733). A wildcard username like user{?} counts as 1 arg, then expands into 3 usernames (user_, user-, user.) inside the query loop. The output path is recomputed and the file reopened with mode "w" on every iteration, so each variant's results overwrite the previous ones — all but the last variant are silently lost.
Steps to reproduce
# offline repro (run from the repo root): drives main() with a stubbed sherlock() so results are deterministic
import sys, os, contextlib, io
sys.path.insert(0, ".")
sys.argv = ["sherlock", "--local", "--txt", "--output", "/tmp/x.txt", "user{?}"]
import sherlock_project.sherlock as S
from sherlock_project.result import QueryResult, QueryStatus
def fake(username, site_data, query_notify, **kw):
url = f"https://example.com/{username}"
return {"Example": {"url_main": "https://example.com",
"url_user": url,
"status": QueryResult(username, "Example", url, QueryStatus.CLAIMED),
"http_status": 200, "response_text": b""}}
S.sherlock = fake
S.requests.get = lambda *a, **k: (_ for _ in ()).throw(RuntimeError("offline"))
os.makedirs("/tmp", exist_ok=True)
with contextlib.redirect_stdout(io.StringIO()):
S.main()
print(open("/tmp/x.txt").read())Observed: /tmp/x.txt contains only the last variant's results (user.), and the Total line reflects only the surviving variant (1, not 3). user_ and user- results are silently discarded. Live runs show the same: only one variant's hits survive in the file.
Expected: either the same single-username error already used for multiple raw usernames, or correct accumulation across variants — not silent truncation.
Root cause
- Guard at
sherlock.py:733runs before{?}expansion and cannot see the expanded count. - The
{?}expansion loop inmain()(~809-814) turns 1 raw arg into 3 usernames. - Loop reopens
result_filewith"w"per iteration (~825-836).
Note
This is adjacent to #2992 (--output ignored without --txt) but distinct: here --txt IS passed and the guard is bypassed entirely by expansion. PR #3083's proposed fix does not close this path (it widens the affected surface). No other open PR covers it.
Environment: macOS, Python 3.13, master @ 3760187 (verified byte-identical to upstream master).
Source: sherlock-project/sherlock