[Feature]: Add NeuG as an optional embedded graph+vector backend (native FTS/HNSW)
Problem Statement
When running cognee locally/embedded (no external Neo4j/LanceDB service), none of the built-in, zero-config backends provide graph + vector(ANN) + native full-text (FTS) in a single engine. Three facts from the dispatch/adapter code:
- Graph — in cognee, the built-in graph backends (
ladybug/kuzu,neo4j,neptune) are wired graph-only: no vector index, no native FTS path. Graph and vector are two separate engines, so the same data gets written twice. (The one graph+vector hybrid,neptune_analytics, is a cloud service — not zero-config.) - Vector — the built-in vector backends (
lancedb/pgvector/turso) build no ANN index. The defaultlancedbstores embeddings but never calls LanceDB's ANN API, sosearch()is a flat exact scan. Engines that do build ANN indexes (qdrant / milvus / weaviate) need a separate service or plugin. - Full-text —
SearchType.CHUNKS_LEXICALruns the in-memoryBM25ChunksRetrieverfor every backend, so none can use a native FTS index.
cognee's core scenario (single-machine AI memory / agent) is exactly "local, zero-service, one engine" — a tier with thin built-in options today. NeuG would be a good fit here: a zero-config embedded backend that eagerly builds HNSW + native FTS indexes.
Proposed Solution
Add NeuG as an optional backend. NeuG is a C++ graph database (Cypher; embedded + service modes) that natively provides:
- Full-text search — index-accelerated lexical retrieval;
- Vector search — a native C++ engine (zvec), currently HNSW.
So a single embedded engine can serve graph + vector + FTS at once, filling the gap above. We already have a working integration on our fork (NeuGGraphAdapter + NeuGVectorAdapter, following cognee's built-in adapter conventions, with unit + integration tests). We're opening this issue to align on the right submission path before sending a PR.
Alternatives Considered
- A cognee-community plugin (
use_graph_adapter/use_vector_adapter) is feasible, but the registry overrides globally bySearchTypeand can't express "native FTS only when backend = NeuG" — soCHUNKS_LEXICALwould still fall back to in-memory BM25, losing the core differentiator. We therefore lean built-in, but welcome maintainers' call on main-repo vs community-repo.
Use Case
- Local/embedded AI memory & agent: one local engine for graph + vector + FTS, no external services, fast lexical retrieval (native FTS index vs in-memory rescan).
- Benchmark (cognee-orchestrated). Quality on LoCoMo-10 (1540 Q;
qwen-plusanswers,qwen-maxjudges); retrieval latency on LongMemEval-M (51661 sessions, ~2.8M-edge graph, measured at the search layer). Baseline = cognee's zero-config default (ladybug/kuzu+lancedb+ in-memory BM25):
| NeuG | default | neo4j | |
|---|---|---|---|
| Quality — accuracy | 0.661 | 0.650 | — |
| Graph multi-hop — p50 latency | 5.45 ms | 17.04 ms | 145.07 ms |
| Vector / FTS / hybrid — speedup vs default | 10.9× / 958× / 5.7× | 1× | — |
Implementation Ideas
- Two adapters (graph + vector) share one NeuG DB, via the existing
elif provider == "neug"factory convention. - One minimal core change: add a provider branch to the
CHUNKS_LEXICALretriever selection (neug→ native FTS; others → the original BM25). Non-NeuG backends are unchanged (graceful degradation). - Add a
neugoptional extra topyproject.toml(neug is pip-installable).
Open questions for maintainers:
- For a new backend like this — main repo built-in or cognee-community? (Native FTS routing is lost on the community-plugin path.)
- The per-backend lexical routing is new at the retrieval layer. If you'd rather generalize it into a backend-declared capability (e.g.
supports_native_fts), we're happy to help — but it touches all backends and should be your design call. - Any requirements for the optional extra's dependencies / packaging?
Source: topoteretes/cognee