Bug: Edge invalidation searches the whole graph, so unrelated facts retire each other
Environment
- Graphiti version: based on 0.29.3. Line numbers below are from a fork with unrelated local patches, so they may be off by a few lines from stock 0.29.3, but the code quoted is unmodified.
- Database backend: FalkorDB
- LLM provider: Google Gemini (small model used for the dedupe/contradiction judge)
Problem
Saving a memory that merely mentions an entity can retire an unrelated, still-true fact about that entity. The retired fact stays in the graph but is stamped with an invalid_at date, so searches downrank or drop it and the knowledge is effectively gone — silently, with no signal that anything was lost.
In one production graph of roughly 3,950 facts, 1,616 (41%) carry an invalid_at. A hand-audit of four of them found three were collateral rather than genuine change:
| Retired fact | Killed by | Verdict |
|---|---|---|
A person holds a particular job title at a company (WORKS_AT person→company) |
The same person administers that company's source-control organization (ResponsibleFor person→org account) |
wrong — additive role, different target entity |
A person is employed by a company, where they manage a compliance function (WORKS_AT person→company) |
The same person imposed an audit requirement (IMPOSED_REQUIREMENT person→requirement) |
wrong — unrelated fact, different target entity |
A company must maintain a security-standard posture (SUBJECT_TO company→posture) |
That company requires a review against the same standard (REQUIRES company→standards review) |
wrong — narrower restatement, different target entity |
helper.py is a script belonging to a plugin (PART_OF .py file→plugin) |
helper.rb is a script belonging to that plugin (PART_OF .rb file→plugin) |
right — the script really was ported to Ruby |
Four is too small a sample to characterize the other 1,612. But the three wrong cases share a shape: the killing edge shares one endpoint with its victim — the same person, the same company — while its other endpoint is a completely different thing, and it carries a different relation name. The one correct case has the opposite shape: a different endpoint, but the same relation name (PART_OF) pointing at the same shared parent.
The third case has a second problem worth naming: "posture" and "standards review" are arguably the same entity and should have merged during node deduplication. That's a separate weakness. It doesn't explain the first two, where the target entities are unmistakably different things.
Why it happens
1. The candidate pool lost its scoping in a refactor. Two searches run per extracted edge in graphiti_core/utils/maintenance/edge_operations.py. Duplicate detection is scoped to edges between the same two nodes (line 399). Invalidation candidates are not scoped at all (line 414):
search(
clients,
extracted_edge.fact,
group_ids=[extracted_edge.group_id],
config=EDGE_HYBRID_SEARCH_RRF,
search_filter=SearchFilters(), # no filter: any edge in the group is eligible
)This looks like an unintended regression. Before #906 the same code path called a purpose-built helper:
get_edge_invalidation_candidates(driver, extracted_edges, SearchFilters(), 0.2)which restricted candidates to edges touching one of the new edge's endpoints:
WHERE n.uuid IN [edge.source_node_uuid, edge.target_node_uuid]
OR m.uuid IN [edge.target_node_uuid, edge.source_node_uuid]#906 routed both searches through the generic search() so they would work across backends — necessary, since the helpers above issue raw Cypher. The duplicate search kept its scoping, because edge_uuids was added to SearchFilters in the same PR. The invalidation search had no equivalent filter to move to, so it went out unscoped and now nominates any semantically similar edge in the entire group. get_edge_invalidation_candidates still exists in graphiti_core/search/search_utils.py and is no longer reachable from ingestion — only from tests. The old min_score of 0.2 was dropped in the same change.
2. Nothing checks the LLM's contradiction call. Both candidate lists go to the LLM as bare fact strings, with no indication of which entities each fact connects (lines 700–713). Whatever comes back in contradicted_facts becomes the invalidation set. From there:
resolve_edge_contradictions(lines 538–573) applies a purely temporal rule — if the candidate became valid earlier than the new edge, retire it. It has no way to tell whether the two facts are about the same thing.- Lines 826–839 apply the mirror rule: if a flagged candidate became valid later, the new edge is written with
invalid_atalready set — born retired.
So one misjudgement by a small model, on facts stripped of their entity context, is acted on in both directions with nothing downstream to catch it. #1666 measures how unreliable that judge is on non-reasoning small models, reporting the opposite symptom — genuine contradictions missed. Both failures point at the same place, and a structural check helps in a way a better prompt cannot: it bounds what a wrong answer can destroy.
The invalid_at stamped on the retired edge is the new edge's valid_at, which usually derives from the ingesting episode's reference time. That's why collateral retirements are dated to the day something was saved rather than the day anything changed — in one ingest, eight separate edges all shared the same valid_at, to the second.
Proposed fix
Restoring that earlier scoping is not enough on its own: "touches either endpoint" still admits all three collateral cases, because each killing edge shares a person or company with its victim.
An edge should only be retired by an edge that could plausibly replace it. Two shapes qualify:
- Same two entities, in either direction. This is a restatement of the same relationship, so a relabeled relation still supersedes correctly — "service
USESdatastore" giving way to "serviceMIGRATED_OFFdatastore". - One shared entity and the same relation name. This is what a rename or a port looks like — the
PART_OFcase above, wherehelper.rblegitimately replaceshelper.pyunder the same parent.
def _relation_key(name: str) -> str:
"""Fold case and separators: `MIGRATED_OFF`, `migrated-off` and `Migrated Off` all match."""
return re.sub(r'[\s_-]+', '', name or '').casefold()
def _could_replace(candidate: EntityEdge, resolved_edge: EntityEdge) -> bool:
"""True when `resolved_edge` is plausibly a replacement for `candidate`.
Either it connects the same two entities, and so can restate that relationship; or it shares
one endpoint and carries the same relation name, which is what a rename or a port looks like.
Without this, any semantically similar edge in the group is eligible, and a new fact about a
person's side project can retire the fact recording their job.
"""
candidate_pair = {candidate.source_node_uuid, candidate.target_node_uuid}
resolved_pair = {resolved_edge.source_node_uuid, resolved_edge.target_node_uuid}
if candidate_pair == resolved_pair:
return True
return bool(candidate_pair & resolved_pair) and _relation_key(
candidate.name
) == _relation_key(resolved_edge.name)Apply it once where the candidate list is assembled (~line 777, after the LLM's contradicted_facts are mapped back to edges):
invalidation_candidates = [
candidate for candidate in invalidation_candidates
if _could_replace(candidate, resolved_edge)
]Filtering here rather than inside resolve_edge_contradictions covers both symptoms — the retire-the-old-edge path and the born-retired-new-edge path share this list.
Against the four audited cases: the three wrong retirements stop and the correct one still fires.
Comparing entity pairs unordered is deliberate. Extraction direction is not stable — "a person works at a company" can come out as person -WORKS_AT-> company or company -EMPLOYS-> person — and a genuine supersession whose direction flipped between ingests should still invalidate.
Trade-offs
- A relation renamed and re-pointed at the same time will no longer invalidate. Both clauses fail, and the old fact lingers as stale. That's the acceptable direction of error: a stale fact that lingers is visible and correctable, while a true fact silently retired is neither.
- Relation-name equality carries real weight now, which makes it sensitive to how consistently the extractor labels relations. Names are not normalized in the graph above (
UsesandUSEScoexist, likewiseAffects/AFFECTS), which is why the comparison folds case and separators rather than comparing raw strings.
Considered: just restoring the earlier helper
Cheaper, and it would cut LLM cost by shrinking the pool. But measured against the audited cases it prevents none of the collateral, so it is not a fix on its own. Worth doing for cost and candidate quality; not a substitute.
There is also nothing to restore it with. SearchFilters has no way to express "edges touching these nodes," which is why #906 dropped the scoping rather than porting it — the duplicate path's scoping survived only because edge_uuids was added in the same PR. The driver layer can already do it (edge_similarity_search accepts source_node_uuid / target_node_uuid), but search() exposes no way to reach it. A node_uuids filter would close that gap for every backend and let the dropped min_score=0.2 come back with it. Better as separate work: it touches each provider's filter constructor, and the guard above is what actually prevents the collateral.
Tests
- Unit-test the filter through
resolve_extracted_edgewith a mocked LLM returningcontradicted_facts, following the existing pattern intests/utils/maintenance/test_edge_operations.py. Cases: same pair with an earliervalid_at→ invalidated; same pair with the direction flipped → invalidated; one shared endpoint and the same relation name → invalidated; one shared endpoint and a different relation name → not invalidated; no shared endpoint → not invalidated. - Cover the born-retired path: a candidate with a later
valid_atthat fails_could_replacemust not setinvalid_aton the new edge. - Cover name folding:
USESandUsesmust be treated as the same relation. - Run the existing suite — this is on the main ingest path. #1603 adds unit tests for
resolve_edge_contradictionsitself; this change leaves that function untouched, so the two should not conflict.
Out of scope
- Repairing existing damage. This change stops further collateral but does not undo any that has already happened. Facts wrongly retired before the fix stay retired, and any deployment that has been ingesting for a while will have some. Identifying and reviving them means deciding, per edge, whether its
invalid_atreflects real change or a bad match — separate work, with its own risk of clearing dates that were correct. - Search-side handling of invalidated facts.
search_memory_factsreturns retired facts interleaved with live ones — across 195 recorded searches in the graph above, 30% of returned facts carried aninvalid_at. Already proposed separately in #1645.
Source: getzep/graphiti