Similarity-evaluation test suite has no polarity cases: a query and its exact reversal clear the 0.8 hit threshold, so the cache serves the answer to the opposite question.

Summary

A semantic cache reuses a stored answer when a new query scores above similarity_threshold (default 0.8) against a cached query. The threshold assumes: high similarity ⇒ same question. For the query pairs a cache must not merge a question and its reversal, as the assumption runs backwards. Further, reversing a query is a minimal edit that keeps almost all tokens, so it scores far above 0.8, while an equivalent query phrased freshly can score below it. The result is the worst trade for a cache: false hits on opposite-meaning queries, misses on same-meaning ones. The similarity-evaluation tests cover paraphrase-like positives; none of them pit shared wording against opposite meaning. This is a test blind spot report, not a claim the cache is broken (no cosine cut separates these classes) which is why the tests should pin the boundary.

Minimal reproducible case

Real specimen pairs from a released audit corpus, cosines measured under nomic-embed-text-v1.5 (256-dim MRL truncation; frozen per-pair results ship with the paper's artifact, and values shift by embedding model, but bottom line is this mutation class scored 0.83–0.9997 across all nine encoder configurations we tested):

• Cached query: "Withhold the study drug from any participant who reports chest tightness." New query: "Administer the study drug to any participant who reports chest tightness." Cosine 0.9608 > 0.8 → cache hit. The user asking about administering receives the cached answer about withholding.
• Cached query: "Limit each household to at most 3 emergency food boxes per week." New query: "Limit each household to at most 30 emergency food boxes per week." Cosine 0.9980 > 0.8 → cache hit across a 10× quantity change.

The mirror case: "Retry the request at most three times." vs. the equivalent "Give the call up to three attempts, then stop." averages cosine ≈ 0.76 under the same encoder → cache miss on a reusable answer.

from gptcache import Config
# default: Config().similarity_threshold == 0.8
# With any embedding-cosine SimilarityEvaluation, the withhold/administer pair
# above evaluates above threshold and returns the cached completion.

Proposal

Add polarity cases to the similarity-evaluation test suite, per §11.3 of the audit paper (published: https://arxiv.org/abs/2608.10216):

1. Query pairs sharing wording, with opposite decisions (negation flip, must→may, quantity/unit changes, scope inversion) asserted as must-NOT-hit fixtures at the default threshold, xfail where they currently hit.
2. Query pairs sharing the decision with no shared wording are documented as the miss-side boundary.

For safety adjacent domains (the clinical pair above is real), a recommendation therein is to disable semantic caching, or require exact match, would follow directly from the fixtures. Happy to contribute the cases as a PR; the corpus is released with the paper's artifact.