[Bug]: retrieval has no relevance floor — every query returns top_k results (distance filtering reverted in #4379)

Summary

Retrieval has no relevance floor at any layer. Vector search is ORDER BY distance LIMIT top_k with no distance predicate, so on a non-empty collection every query returns top_k results, no matter how unrelated the query is to the corpus. There is no way — via the SDK, the MCP recall tool, or config — to express "return nothing if nothing is relevant."

The practical consequence for agent-facing use: CHUNKS returns confident-looking garbage, and GRAPH_COMPLETION feeds that garbage to an LLM with no signal that it is low-relevance context, which is a direct hallucination path.

I'd like to contribute a fix, but the history here (#3006 → reverted in #4379) suggests the maintainers have a view on the right shape, so I'm opening an issue rather than a PR. Guidance welcome.

Reproduction

Corpus: a few hundred chunks of purely technical material — software engineering and system administration notes. No biological or natural-science content whatsoever.

search_type=CHUNKS, top_k=3
query: "optimal breeding cycle of Antarctic emperor penguins"

Returns 3 chunks at full confidence, each an unrelated technical note on a different topic. Correct behaviour would be an empty result or an explicit "no relevant context" signal.

Any corpus reproduces this: ingest any documents on one subject, then query a subject the corpus does not cover. You always get top_k results back.

This is not a bad-embedding problem — the ranking is fine, those genuinely are the nearest neighbours. The problem is that nearest is unconditionally treated as relevant.

Root cause

Verified in cognee==1.4.2 (image cognee/cognee:main, commit 4b9dd362625dfd3621c344e571a86f5bc7a55ee8) and re-checked against current main.

grep -rniE 'threshold|min_score|score_cutoff|distance_threshold' over cognee/modules/retrieval/ and cognee/infrastructure/databases/vector/ returns zero matches.

1. Vector layer — PGVectorAdapter.search() (cognee/infrastructure/databases/vector/pgvector/PGVectorAdapter.py:470):

query = select(
    *select_columns,
    PGVectorDataPoint.c.vector.cosine_distance(query_vector).label("similarity"),
).order_by("similarity")

if limit > 0:
    query = query.limit(limit)

ORDER BY + LIMIT, no WHERE on distance. Semantics are "k nearest", never "k similar". In a non-empty collection the k nearest always exist.

2. Retriever layer — ChunksRetriever.get_retrieved_objects() passes limit=self.top_k and returns whatever comes back. The only empty-result path is CollectionNotFoundError → NoDataError, i.e. an empty result means no data ingested, never nothing relevant.

3. API/MCP layer — neither the retriever constructors nor the MCP recall tool expose any threshold parameter; top_k is the sole knob.

Note the raw distance is already computed and carried: PGVectorAdapter sets score to the raw cosine distance (# Return backend raw cosine distance as score (lower is better)). The information needed to filter is present and correctly scaled — it is simply never used as a predicate.

Prior art in this repo

• #2720 (closed) — "returns identical subgraph regardless of query." Same underlying cause, observed through the graph path: when wide_search_top_k (default 100) exceeds collection size, vector search returns everything and the graph filter degenerates.
• #3006 (merged 2026-06-07) — added max_distance (default 1.5) to extract_relevant_node_ids(), fixing #2720.
• #4379 (merged 2026-08-08) — reverted #3006 with no stated rationale. max_distance is absent from main today, so #2720's cause is live again.

Guessing at the revert: a fixed default cosine cutoff (1.5) applied inside graph traversal is hard to justify across embedding models and silently drops recall for existing users. If that's the concern, I'd agree — which is why the proposal below defaults to off.

• #3786 (open) — Pillar A: adds relevance, citations, and a Confidence enum including ABSTAIN. That's the closest thing to a fix, but it is advisory: it labels a weak result without preventing retrieval from returning it, and (as of now) wires up graph completion only. A caller still cannot ask for "nothing unless it clears this bar." The two are complementary, not redundant: #3786 reports confidence, this asks for enforcement.

Proposed fix

Opt-in, off by default, so no existing behaviour changes unless requested.

1. Optional max_distance: Optional[float] = None on VectorDBInterface.search(), implemented per adapter as a WHERE predicate (for pgvector, .where(cosine_distance(...) < max_distance)) so filtering happens in the DB rather than in Python, keeping LIMIT meaningful.
2. Plumb an optional max_distance through the retriever constructors and expose it on search() / MCP recall.
3. When everything is filtered out, return an empty result — and for completion-style retrievers emit an explicit "no relevant context found" rather than prompting the LLM with unrelated chunks.
4. None = current behaviour exactly.

This keeps the distance semantics at the layer that owns them (each adapter knows its metric) instead of hardcoding a cosine constant in traversal code, which I suspect is what sank #3006.

Two open questions for maintainers:

• Absolute vs. relative. An absolute cutoff needs per-embedding-model calibration (a good cosine threshold for bge-m3 isn't one for OpenAI embeddings). A relative rule ("drop results whose distance exceeds 1.3× the best hit") is model-agnostic but won't reject a uniformly-irrelevant result set, which is exactly the case in the repro above. My inclination is absolute, opt-in, documented as requiring calibration — but a relative_margin could be added alongside.
• Should defaults ever change? I'd say no, given #4379. Opt-in only, with the recommended value documented per common embedding model.

Environment

• cognee 1.4.2, image cognee/cognee:main @ 4b9dd36
• pgvector (pgvector/pgvector:pg17), embeddings bge-m3 via a LiteLLM-compatible custom provider
• Reproduced through both the MCP recall tool and direct retriever calls

Happy to open a PR along the lines above if the shape sounds right — in particular I'd want confirmation on the absolute-vs-relative question and on whether the adapter-level WHERE is the preferred home for this, before writing code.