core: `InMemoryVectorStore` raises `NotImplementedError` for relevance-score search and the `similarity_score_threshold` retriever
Submission checklist
- This is a bug, not a usage question.
- I added a clear and descriptive title that summarizes this issue.
- I used the GitHub search to find a similar question and didn't find it.
- I am sure that this is a bug in LangChain rather than my code.
- The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).
- This is not related to the langchain-community package.
- I posted a self-contained, minimal, reproducible example. A maintainer can copy it and run it AS IS.
Package (Required)
- langchain
- langchain-openai
- langchain-anthropic
- langchain-classic
- langchain-core
- langchain-model-profiles
- langchain-tests
- langchain-text-splitters
- langchain-chroma
- langchain-deepseek
- langchain-exa
- langchain-fireworks
- langchain-groq
- langchain-huggingface
- langchain-mistralai
- langchain-nomic
- langchain-ollama
- langchain-openrouter
- langchain-perplexity
- langchain-qdrant
- langchain-xai
- Other / not sure / general
Related Issues / PRs
No response
Reproduction Steps / Example Code (Python)
import asyncio
from langchain_core.embeddings import DeterministicFakeEmbedding
from langchain_core.vectorstores import InMemoryVectorStore
vector_store = InMemoryVectorStore(DeterministicFakeEmbedding(size=8))
vector_store.add_texts(["foo", "bar"])
retriever = vector_store.as_retriever(
search_type="similarity_score_threshold",
search_kwargs={"score_threshold": 0.5},
)
for label, call in [
("similarity_search_with_relevance_scores", lambda: vector_store.similarity_search_with_relevance_scores("foo", k=1)),
("retriever.invoke", lambda: retriever.invoke("foo")),
("retriever.ainvoke", lambda: asyncio.run(retriever.ainvoke("foo"))),
]:
try:
print(label, "->", call())
except NotImplementedError as e:
print(label, "-> raised", repr(e))
# Actual (all three): raised NotImplementedError()
# Expected: (Document, score) pairs with scores in [0, 1], filtered by score_threshold
Error Message and Stack Trace (if applicable)
Description
VectorStore.as_retriever and VectorStore.search advertise search_type="similarity_score_threshold", and VectorStoreRetriever accepts that configuration at construction time. Both paths go through similarity_search_with_relevance_scores / asimilarity_search_with_relevance_scores, which call self._select_relevance_score_fn(). The base implementation of _select_relevance_score_fn raises NotImplementedError and says vector stores should define their own.
InMemoryVectorStore, the reference implementation shipped in langchain-core, never overrides it, so relevance-score search and the threshold retriever fail with a bare NotImplementedError (sync and async) as soon as they are invoked — even though the retriever was constructed without complaint and the store already computes cosine similarity for every result.
Note that InMemoryVectorStore.similarity_search_with_score returns cosine similarity (range [-1, 1], higher is better), not a distance, so the existing VectorStore._cosine_relevance_score_fn (which computes 1 - distance) can't be reused as-is — it would invert the ranking.
Proposed fix: override _select_relevance_score_fn in InMemoryVectorStore to normalize cosine similarity into [0, 1] (e.g. lambda score: (score + 1) / 2), and add sync and async tests for similarity_search_with_relevance_scores and the similarity_score_threshold retriever. I'd be glad to open a PR with the fix and tests if a maintainer assigns this to me.
System Info
- langchain-core 1.6.3 (also current
master) - Python 3.10.16
- macOS 26.6 (arm64)
Social handles (optional)
No response
Source: langchain-ai/langchain