Context Search returns negative loss scores inconsistent with the documented formula

Author: leemeiiCreated Sep 11, 2026Updated Sep 14, 2026

Labelsbug

Summary

Qdrant 1.19.0 applies an additional nonlinear compression to negative Context Search losses, causing the returned scores to differ from the formula documented by Qdrant.

The documented formula is:

sum(min(positive_similarity - negative_similarity, 0.0))

However, when the raw loss is -1.6, Qdrant returns approximately -0.61538464. When the raw loss is -0.4, it returns approximately -0.28571436.

These values correspond to an additional transformation:

x / (1 + abs(x))

rather than the documented raw loss.

Environment

Qdrant: qdrant/qdrant:v1.19.0
Distance: Cosine
Vector dimension: 2
API: POST /collections/{collection_name}/points/query
Query type: Context Search
Search mode: exact=true
Filter: none
Quantization: disabled
Python dependency: requests==2.34.2

Steps to Reproduce

Start Qdrant:

bash

docker run --rm --name qdrant-context-score-repro \
  -p 127.0.0.1:16354:6333 \
  qdrant/qdrant:v1.19.0

Install the dependency:

bash

pip install requests==2.34.2

Run this standalone Python script:

python

import json
import math
import uuid
import requests

BASE = "http://127.0.0.1:16354"
COLLECTION = "context_score_" + uuid.uuid4().hex[:8]

POSITIVE = [1.0, 0.0]
NEGATIVE = [-1.0, 0.0]

SEED = {
    0: [0.8, 0.6],
    1: [0.0, 1.0],
    2: [-0.8, 0.6],
    3: [-0.2, 0.9797958971],
    4: [0.3, -0.9539392014],
}

MUTANT = [-0.5, 0.8660254038]


def call(method, path, body=None):
    response = requests.request(
        method,
        BASE + path,
        json=body,
        timeout=30,
    )
    response.raise_for_status()
    return response.json() if response.content else {}


def dot(left, right):
    return sum(a * b for a, b in zip(left, right))


def documented_score(vector):
    positive = dot(POSITIVE, vector)
    negative = dot(NEGATIVE, vector)
    return min(positive - negative, 0.0)


def query():
    result = call(
        "POST",
        f"/collections/{COLLECTION}/points/query",
        {
            "query": {
                "context": [
                    {
                        "positive": 100,
                        "negative": 101,
                    }
                ]
            },
            "limit": len(SEED),
            "params": {
                "exact": True,
            },
        },
    )["result"]["points"]

    return {
        int(point["id"]): float(point["score"])
        for point in result
    }


try:
    call(
        "PUT",
        f"/collections/{COLLECTION}",
        {
            "vectors": {
                "size": 2,
                "distance": "Cosine",
            }
        },
    )

    points = [
        {"id": 100, "vector": POSITIVE},
        {"id": 101, "vector": NEGATIVE},
    ]

    points.extend(
        {"id": point_id, "vector": vector}
        for point_id, vector in SEED.items()
    )

    call(
        "PUT",
        f"/collections/{COLLECTION}/points?wait=true",
        {"points": points},
    )

    seed_actual = query()
    seed_expected = {
        point_id: documented_score(vector)
        for point_id, vector in SEED.items()
    }

    call(
        "PUT",
        f"/collections/{COLLECTION}/points/vectors?wait=true",
        {
            "points": [
                {
                    "id": 1,
                    "vector": MUTANT,
                }
            ]
        },
    )

    mutant_actual = query()
    mutant_expected = dict(seed_expected)
    mutant_expected[1] = documented_score(MUTANT)

    print(json.dumps({
        "seed_actual": seed_actual,
        "seed_expected_documented": seed_expected,
        "mutant_actual": mutant_actual,
        "mutant_expected_documented": mutant_expected,
    }, indent=2, sort_keys=True))

    assert abs(seed_actual[2] - seed_expected[2]) > 1e-5
    assert abs(mutant_actual[1] - mutant_expected[1]) > 1e-5

    print("BUG REPRODUCED")

finally:
    requests.delete(
        f"{BASE}/collections/{COLLECTION}",
        timeout=30,
    )

Actual Behavior

Typical output:

seed_actual:
{
  0: 0.0,
  1: -0.0000001192,
  2: -0.61538464,
  3: -0.28571436,
  4: 0.0
}

seed_expected_documented:
{
  0: 0.0,
  1: 0.0,
  2: -1.6,
  3: -0.4,
  4: 0.0
}

After updating point 1:

mutant_actual[1] = -0.50000006
mutant_expected_documented[1] = -1.0

The returned values correspond to:

-1.6 / (1 + 1.6) = -0.6153846
-0.4 / (1 + 0.4) = -0.2857143
-1.0 / (1 + 1.0) = -0.5

Expected Behavior

Returned scores should exactly follow the documented Context Search formula:

score = sum(min(positive_similarity - negative_similarity, 0.0))

Expected values:

seed:
point 0:  0.0
point 1:  0.0
point 2: -1.6
point 3: -0.4
point 4:  0.0

after mutation:
point 1: -1.0

Possible Solution

Do not apply an additional nonlinear compression to each negative loss, or update the official documentation to clearly state that the API returns compressed scores rather than the raw documented loss.

If the compressed score is intentional, consider exposing the raw Context Search loss separately so clients can interpret the result according to the documented formula.

Context (Environment)

This affects applications that use Context Search scores for:

score thresholds;
score calibration;
cross-query score comparison;
combining multiple scoring signals;
recommendation or ranking decisions based on loss magnitude.

Non-negative losses still return 0.0, so this is not an empty-result or result-order issue. It is a score-semantics mismatch for negative losses.

Detailed Description

The positive context vector is:

[1.0, 0.0]

The negative context vector is:

[-1.0, 0.0]

For candidate [-0.8, 0.6]:

positive_similarity = -0.8
negative_similarity = 0.8
difference = -1.6

The documented score is -1.6, but the API returns -0.61538464.

For candidate [-0.2, 0.9797958971]:

difference = -0.4

The documented score is -0.4, but the API returns -0.28571436.

The behavior reproduces across fresh collections and remains after updating a stored vector.

Possible Implementation

Add Context Search regression tests covering:

zero loss;
negative loss;
vector updates;
Cosine, Dot, and Manhattan distance metrics.

The tests should verify that the API output matches the documented formula, or explicitly document any intentional score transformation.

Source: qdrant/qdrant

View original on GitHub View discussion on GitHub