[Bug]: untracked files are invisible to the graph — discovery is git ls-files only, so a newly written file is never indexed, even by a full rebuild

Summary

File discovery is sourced entirely from git ls-files and git diff, so a file that git has never been told about is invisible to the graph — not just until the next build, but permanently, including through a full rebuild. For an AI assistant that writes a new file and immediately asks the graph about it, this is the common case: the assistant's own new code is the one thing the graph cannot see.

Where

code_review_graph/incremental.py, both discovery paths:

• Full build — collect_all_files() takes its candidate list from get_all_tracked_files(), which is git ls-files. Untracked files never become candidates, so full_build() never parses them. Worse, full_build()'s stale purge computes existing_files - current_abs from that same list, so an untracked file that somehow reached the graph is actively deleted from it on the next rebuild.
• Incremental update — incremental_update() gets its change set from get_changed_files(), which is git diff --name-status <base> --. git diff does not report untracked paths, so the file never enters all_files and is never parsed.

get_staged_and_unstaged() does see untracked files (git status --porcelain), but it is only reached by detect-changes, update --brief and the review tools as a fallback when get_changed_files() returns empty — never by the code that actually writes nodes into the graph.

Reproduction

In any repo with at least one commit:

code-review-graph build                      # note the file count
printf 'def marker_fn():\n    return 1\n' > new_module.py   # do NOT git add
code-review-graph update                     # "No changes detected"
code-review-graph build                      # file count unchanged
# query_graph_tool(query_type="callers_of", target="marker_fn") -> not_found

git add -N new_module.py                     # intent-to-add, no content change
code-review-graph build                      # file count +1, marker_fn resolves

git add -N changes nothing about the file's content — only git's index — which isolates the cause to discovery.

Impact

The MCP integration is wired to be live: a PostToolUse hook runs code-review-graph update after every edit, and serve --auto-watch watches the tree. Both are defeated for new files. The assistant is told the graph is current, asks callers_of / references_to / file_summary about code it wrote seconds ago, gets zero, and concludes the symbol does not exist.

This is the same failure #987 describes, approached from the other end: #987 makes the empty result admit its uncertainty, which is a real improvement, but the result is still empty. The proposal here is that it should not be empty.

Proposed fix

Make discovery tracked ∪ untracked-but-not-ignored, which is exactly git ls-files ∪ git ls-files --others --exclude-standard. --exclude-standard applies .gitignore, .git/info/exclude and core.excludesFile, so build output, node_modules, virtualenvs and the tool's own .code-review-graph/ stay out — the existing DEFAULT_IGNORE_PATTERNS pass then applies on top as it does today. Unlike git status --porcelain, ls-files --others lists individual files rather than collapsing a wholly-untracked directory to dir/, so no second walk is needed.

Two call sites:

• collect_all_files() — union the untracked list into candidates (keep the filesystem-walk fallback for when both are empty, i.e. not a git repo).
• get_changed_files() — union the untracked list into the git branch's result, so the incremental path sees new files. A file git has never seen is, by any useful definition, a change.

--recurse-submodules is not valid with --others, so the submodule option applies only to the tracked call.

Deletion caveat — also fixed in the local patch

Once untracked files are indexed, deleting one is invisible to the incremental path in the same way: git diff never names it (never tracked) and --others no longer names it (gone), so remove_file_data() is not called and the node lingers until the next full build, whose stale purge does clear it. serve --auto-watch catches it via filesystem events, so this only affects hook-only setups (the PostToolUse hook calling update, or any other caller of incremental_update() with no watcher running).

Update: this is no longer an open gap in my local patch. incremental_update() now resolves the graph's recorded file list against disk before its early return: for every file the graph already knows about that this update's own diff/untracked scan didn't name, it stats the path and calls the existing remove_file_data() if the file is gone — the same mechanism the watcher's on_deleted uses, just triggered by a filesystem stat instead of a filesystem event. Verified round-trip on a scratch repo: create an untracked file → update → node appears; delete it, no rebuild → update → node is gone. No extra git calls, no watcher required.

Environment

• code-review-graph 2.3.6 (installed as a uv tool), macOS 15, Python 3.13
• Verified still present on main as of today: collect_all_files() and get_changed_files() are unchanged in this respect in 2.3.8 and later.