#40502·langchain

core: `InMemoryVectorStore` raises `NotImplementedError` for relevance-score search and the `similarity_score_threshold` retriever

Author: mrutunjay-kinagiCreated Sep 16, 2026Updated Sep 17, 2026
Labelsbugcoreexternal

Submission checklist

  • This is a bug, not a usage question.
  • I added a clear and descriptive title that summarizes this issue.
  • I used the GitHub search to find a similar question and didn't find it.
  • I am sure that this is a bug in LangChain rather than my code.
  • The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).
  • This is not related to the langchain-community package.
  • I posted a self-contained, minimal, reproducible example. A maintainer can copy it and run it AS IS.

Package (Required)

  • langchain
  • langchain-openai
  • langchain-anthropic
  • langchain-classic
  • langchain-core
  • langchain-model-profiles
  • langchain-tests
  • langchain-text-splitters
  • langchain-chroma
  • langchain-deepseek
  • langchain-exa
  • langchain-fireworks
  • langchain-groq
  • langchain-huggingface
  • langchain-mistralai
  • langchain-nomic
  • langchain-ollama
  • langchain-openrouter
  • langchain-perplexity
  • langchain-qdrant
  • langchain-xai
  • Other / not sure / general

Related Issues / PRs

No response

Reproduction Steps / Example Code (Python)

import asyncio

from langchain_core.embeddings import DeterministicFakeEmbedding
from langchain_core.vectorstores import InMemoryVectorStore

vector_store = InMemoryVectorStore(DeterministicFakeEmbedding(size=8))
vector_store.add_texts(["foo", "bar"])

retriever = vector_store.as_retriever(
    search_type="similarity_score_threshold",
    search_kwargs={"score_threshold": 0.5},
)

for label, call in [
    ("similarity_search_with_relevance_scores", lambda: vector_store.similarity_search_with_relevance_scores("foo", k=1)),
    ("retriever.invoke", lambda: retriever.invoke("foo")),
    ("retriever.ainvoke", lambda: asyncio.run(retriever.ainvoke("foo"))),
]:
    try:
        print(label, "->", call())
    except NotImplementedError as e:
        print(label, "-> raised", repr(e))

# Actual (all three):  raised NotImplementedError()
# Expected: (Document, score) pairs with scores in [0, 1], filtered by score_threshold

Error Message and Stack Trace (if applicable)

Description

VectorStore.as_retriever and VectorStore.search advertise search_type="similarity_score_threshold", and VectorStoreRetriever accepts that configuration at construction time. Both paths go through similarity_search_with_relevance_scores / asimilarity_search_with_relevance_scores, which call self._select_relevance_score_fn(). The base implementation of _select_relevance_score_fn raises NotImplementedError and says vector stores should define their own.

InMemoryVectorStore, the reference implementation shipped in langchain-core, never overrides it, so relevance-score search and the threshold retriever fail with a bare NotImplementedError (sync and async) as soon as they are invoked — even though the retriever was constructed without complaint and the store already computes cosine similarity for every result.

Note that InMemoryVectorStore.similarity_search_with_score returns cosine similarity (range [-1, 1], higher is better), not a distance, so the existing VectorStore._cosine_relevance_score_fn (which computes 1 - distance) can't be reused as-is — it would invert the ranking.

Proposed fix: override _select_relevance_score_fn in InMemoryVectorStore to normalize cosine similarity into [0, 1] (e.g. lambda score: (score + 1) / 2), and add sync and async tests for similarity_search_with_relevance_scores and the similarity_score_threshold retriever. I'd be glad to open a PR with the fix and tests if a maintainer assigns this to me.

System Info

  • langchain-core 1.6.3 (also current master)
  • Python 3.10.16
  • macOS 26.6 (arm64)

Social handles (optional)

No response