Changed-files incremental rebuild drops cross-file dependency edges (hook install / watch affected)

Summary

The changed-files incremental rebuild (watch._rebuild_code(changed_paths=[...])) is not edge-equivalent to a full rebuild. It drops cross-file dependency edges — including edges belonging to files that were never touched — so dependency, dependent and impact queries under-report after an incremental refresh.

This is reachable from two shipped features, not just the internal API:

• graphify hook install — the post-commit hook calls _rebuild_code(_root, changed_paths=changed, force=_force) (hooks.py:141)
• graphify watch <path> — calls _rebuild_code(..., changed_paths=merged, ...) (watch.py:1067)

So a repository using the git hooks or the watcher accumulates edge loss on every commit or save, silently. The query most affected is graphify affected "X", since it is reverse traversal over exactly the relations being lost.

graphify update is unaffected: it always calls _rebuild_code(watch_path, force=..., no_cluster=..., block_on_lock=True) with changed_paths left at None (cli.py:2151), i.e. a full corpus rebuild. That is likely why this has gone unnoticed — the CLI never exercises the incremental path.

Version: 0.9.47 (graphifyy 0.9.47), macOS, Python 3.14

How it was measured

The comparison that matters is incremental vs a full rebuild of the same working tree, not incremental vs the pre-edit graph. Comparing against the pre-edit graph only shows that the edited file changed, which is expected and correct.

Procedure, on a copy of a 491-file TypeScript/JavaScript repository (346 files indexed):

1. _rebuild_code(root, changed_paths=None) → baseline
2. append one exported function to a single indexed file
3. _rebuild_code(root, changed_paths=[that_file]) → incremental result
4. _rebuild_code(root, changed_paths=None, force=True) → correct result for the same tree
5. diff the edge sets of (3) and (4)

Results

Touched file	Edges missing vs full	…originating from untouched files
web/components/Workspace.tsx	286	259
web/app/layout.tsx	262	261
web/emails/templates.tsx	263	263

Relation breakdown for the Workspace.tsx case (286 total):

imports_from     258
imports           24
references         3
dynamic_import     1

Two details that should help localise the fix:

• Zero edges were invented. The incremental result is a strict subset of the full result. That points at a missing re-resolution pass rather than corruption, and suggests the fix is additive.
• The loss is dominated by edges whose source_file is not the file that changed. Cross-file import resolution looks like a whole-corpus pass that is not re-run for preserved nodes, so their outbound import edges are lost when the graph is reassembled.

Node counts stay almost exactly right (2307 → 2307 in one case, with the new symbol correctly present), which is why the problem is easy to miss: the graph looks healthy by node count and the edited file's own symbols are updated correctly.

Why this matters

The loss is one-directional — it only ever removes dependency edges. So affected and any dependency/dependent query answers "nothing depends on this" about code that does have dependents. For an impact-analysis tool that is the dangerous direction: a change author is told it is safe to change something that is not.

Performance context

The incremental path is genuinely valuable, which is why this seems worth fixing rather than removing:

	Duration
Full rebuild	3.93 s
Incremental, no files changed	0.08 s (~49×, byte-identical graph, zero delta)
Incremental, one file changed	1.34–1.46 s (~3×)

The no-change case is exactly correct, so the preservation machinery itself works; it is specifically cross-file relationship re-resolution that is missing.

Expected

Either of:

1. A changed-files rebuild that re-resolves cross-file relationships for preserved nodes, so the result is edge-equivalent to a full rebuild; or
2. Documentation that changed_paths is not correctness-preserving for dependency analysis — in which case hook install and watch should probably warn, since they use it by default.

Additionally, it would be useful to expose a correctness-preserving changed-files update through the CLI (e.g. graphify update --changed <paths>), so downstream tools can use the fast path without depending on a private function.

Reproduction

Self-contained script, run with the installed venv's interpreter. It performs the four steps above and exits non-zero while the fast path is lossy:

graphify_incremental_probe.py

import collections, json, pathlib, sys, time


def _graph(out):
    payload = json.loads(out.read_text(encoding="utf-8"))
    nodes = {n["id"]: n for n in payload.get("nodes", [])}
    links = {
        (l["source"], l["target"], l.get("relation")): l
        for l in payload.get("links", payload.get("edges", []))
    }
    return nodes, links


def main():
    if len(sys.argv) != 3:
        print("usage: probe.py <repo-copy> <relative/file.ts>")
        return 2
    from graphify.watch import _rebuild_code

    root = pathlib.Path(sys.argv[1]).resolve()
    target = root / sys.argv[2]
    out = root / "graphify-out" / "graph.json"

    t = time.perf_counter()
    _rebuild_code(root, changed_paths=None, no_cluster=True)
    baseline = time.perf_counter() - t
    bn, bl = _graph(out)
    print(f"1. full baseline   {baseline:6.2f}s nodes={len(bn)} edges={len(bl)}")

    original = target.read_text(encoding="utf-8")
    try:
        target.write_text(original + "\n\nexport function __probe() { return 42; }\n",
                          encoding="utf-8")
        t = time.perf_counter()
        _rebuild_code(root, changed_paths=[target], no_cluster=True)
        inc = time.perf_counter() - t
        in_n, in_l = _graph(out)
        print(f"2. changed-files   {inc:6.2f}s nodes={len(in_n)} edges={len(in_l)}")

        t = time.perf_counter()
        _rebuild_code(root, changed_paths=None, no_cluster=True, force=True)
        full = time.perf_counter() - t
        fu_n, fu_l = _graph(out)
        print(f"3. full, same tree {full:6.2f}s nodes={len(fu_n)} edges={len(fu_l)}")
    finally:
        target.write_text(original, encoding="utf-8")

    lost = {k: v for k, v in fu_l.items() if k not in in_l}
    invented = {k for k in in_l if k not in fu_l}
    touched = str(target.relative_to(root)).replace("\\", "/")
    from_touched = sum(1 for v in lost.values() if v.get("source_file") == touched)

    print(f"\nspeedup {full / inc:.1f}x")
    print(f"edges LOST by the fast path : {len(lost)}")
    print(f"  from the touched file     : {from_touched}")
    print(f"  from untouched files      : {len(lost) - from_touched}")
    print(f"edges INVENTED              : {len(invented)}")
    print("  by relation:",
          collections.Counter(v.get("relation") for v in lost.values()).most_common(6))
    return 1 if lost else 0


if __name__ == "__main__":
    raise SystemExit(main())

Run as:

<venv>/bin/python graphify_incremental_probe.py /path/to/repo-copy web/components/Workspace.tsx

Use a copy — it rewrites the named file and rebuilds graphify-out/ in place.

Sample output:

1. full baseline     4.24s nodes=2307 edges=5072
2. changed-files     1.46s nodes=2307 edges=4787
3. full, same tree   4.55s nodes=2308 edges=5073

speedup 3.1x
edges LOST by the fast path : 286
  from the touched file     : 27
  from untouched files      : 259
edges INVENTED              : 0
  by relation: [('imports_from', 258), ('imports', 24), ('references', 3), ('dynamic_import', 1)]

Happy to test a patch against the same repository if that would help.