LanceDBVectorStore.delete(), delete_nodes(), and get_nodes() build invalid SQL predicates — broke after lancedb#3825 changed double-quote parsing
Bug Description
Three methods in llama_index/vector_stores/lancedb/base.py build SQL delete/filter predicates by concatenating raw ID values inside double quotes (doc_id = "item1"). This isn't a longstanding bug — it used to work. Before lancedb/lancedb#3825 (merged 2026-08-26), Lance's SQL planner treated double-quoted tokens as string literals "for compatibility" (that PR's own words). #3825 correctly fixed a real bug (lancedb#2057 — mixed-case column names in double quotes were wrongly read as literals) by making Lance follow standard SQL rules: double quotes now mean identifier, single quotes mean string literal. Right fix on Lance's side, but a breaking change for anyone relying on the old lenient parsing — including this package. doc_id = "item1" is now parsed as "where doc_id equals the column named item1," which doesn't exist, so it errors instead of matching rows. delete_ref_doc() now fails unconditionally, for every input.
Affects three methods identically: delete(), delete_nodes(), and get_nodes() (when called with node_ids).
lancedb 0.38.0 (Aug 31) is the first release with the new behavior. This package declares lancedb>=0.21.1 with no upper bound, so a fresh install today resolves to a broken combination — but this package's own committed uv.lock still pins 0.37.1, which is why make test doesn't currently catch it. Bumping the lockfile alone (no code change) makes test_delete, test_delete_nodes, and test_get_nodes fail immediately with the existing test suite.
Suggested fix: use LanceDB's type-safe expression API (lancedb.expr.col/lit, added in lancedb#3150) instead of hand-built SQL strings, which sidesteps the quoting question entirely:
from lancedb.expr import col, lit
self.table.delete(col(self.doc_id_key) == lit(ref_doc_id))
I have a PR ready with this fix, a new regression test, and the lockfile bump.
AI disclosure (per CONTRIBUTING.md): this bug was found and the fix verified with AI assistance (Claude). Verification: reproduced the failure directly against a raw lancedb table with no LlamaIndex involved; confirmed the regression concretely by bumping this package's own lockfile and running its existing test suite (not assumed); confirmed the fix against normal IDs, an ID containing an embedded single quote, and an injection-shaped ID (correctly matches nothing rather than deleting everything); traced the root cause through llama_index's and lancedb's git/PR history to confirm this is a recent regression, not a longstanding bug.
Version
llama-index-vector-stores-lancedb 0.6.0 (also confirmed present on main)
Steps to Reproduce
No LlamaIndex needed to reproduce — isolates this to LanceDB predicate syntax:
import lancedb, pyarrow as pa
db = lancedb.connect("/tmp/x")
tbl = db.create_table("t", pa.table({"id": ["a"], "doc_id": ["item1"]}))
tbl.delete('doc_id = "item1"')
Or against this package directly: bump lancedb to >=0.38 in this package's environment and run pytest tests/test_vector_stores_lancedb.py — test_delete, test_delete_nodes, and test_get_nodes fail with no other changes.
Relevant Logs/Tracebacks
ValueError: Invalid input, Schema error: No field named item1. Valid fields are id, doc_id, vector, text, metadata, ...
Source: run-llama/llama_index