#3114·sherlock

bug: --output with a {?}-wildcard username silently overwrites the results file (data loss)

Author: twelfthlaborCreated Sep 10, 2026Updated Sep 10, 2026

Summary

--output FILE is guarded to single-username use, but the guard counts raw CLI args (len(args.username) at sherlock_project/sherlock.py:733). A wildcard username like user{?} counts as 1 arg, then expands into 3 usernames (user_, user-, user.) inside the query loop. The output path is recomputed and the file reopened with mode "w" on every iteration, so each variant's results overwrite the previous ones — all but the last variant are silently lost.

Steps to reproduce

python
# offline repro (run from the repo root): drives main() with a stubbed sherlock() so results are deterministic
import sys, os, contextlib, io
sys.path.insert(0, ".")
sys.argv = ["sherlock", "--local", "--txt", "--output", "/tmp/x.txt", "user{?}"]
import sherlock_project.sherlock as S
from sherlock_project.result import QueryResult, QueryStatus
def fake(username, site_data, query_notify, **kw):
    url = f"https://example.com/{username}"
    return {"Example": {"url_main": "https://example.com",
        "url_user": url,
        "status": QueryResult(username, "Example", url, QueryStatus.CLAIMED),
        "http_status": 200, "response_text": b""}}
S.sherlock = fake
S.requests.get = lambda *a, **k: (_ for _ in ()).throw(RuntimeError("offline"))
os.makedirs("/tmp", exist_ok=True)
with contextlib.redirect_stdout(io.StringIO()):
    S.main()
print(open("/tmp/x.txt").read())

Observed: /tmp/x.txt contains only the last variant's results (user.), and the Total line reflects only the surviving variant (1, not 3). user_ and user- results are silently discarded. Live runs show the same: only one variant's hits survive in the file.

Expected: either the same single-username error already used for multiple raw usernames, or correct accumulation across variants — not silent truncation.

Root cause

  • Guard at sherlock.py:733 runs before {?} expansion and cannot see the expanded count.
  • The {?} expansion loop in main() (~809-814) turns 1 raw arg into 3 usernames.
  • Loop reopens result_file with "w" per iteration (~825-836).

Note

This is adjacent to #2992 (--output ignored without --txt) but distinct: here --txt IS passed and the guard is bypassed entirely by expansion. PR #3083's proposed fix does not close this path (it widens the affected surface). No other open PR covers it.

Environment: macOS, Python 3.13, master @ 3760187 (verified byte-identical to upstream master).

Source: sherlock-project/sherlock