[BUG] Retriever results are ranked by string comparison, not numeric similarity

Required prerequisites

• I have read the documentation https://camel-ai.github.io/camel/camel.html.
• I have searched the Issue Tracker and Discussions that this hasn't already been reported. (+1 or comment there if it has.)
• Consider asking first in a Discussion. This is a specific defect with a reproduction, so the tracker fits it better than a discussion.

What version of camel are you using?

0.2.91a7, at 8c791b7 on master.

System information

3.12.3 Linux-7.0.0-1012-aws-x86_64-with-glibc2.39
0.2.91a7

Problem description

VectorRetriever.query stores the score as text, at camel/retrievers/vector_retriever.py:256:

'similarity score': str(result.similarity),

Two places then rank on that field without converting it back:

• camel/retrievers/hybrid_retrival.py:222
• camel/retrievers/auto_retriever.py:268

sorted(with_score, key=lambda x: x['similarity score'], reverse=True)

That is a lexicographic comparison, so the ranking disagrees with the numeric one whenever text order and numeric order differ. Two cases do:

>>> vals = ['0.9', '1e-05', '0.5', '-0.1', '-0.5']
>>> sorted(vals, reverse=True)
['1e-05', '0.9', '0.5', '-0.5', '-0.1']
>>> sorted(vals, key=float, reverse=True)
['0.9', '0.5', '1e-05', '-0.1', '-0.5']

str(1e-05) is '1e-05', and '1' sorts above '0', so a near-zero score outranks every other result. That needs no negative score at all, only a similarity_threshold low enough to admit a small positive one. The default is 0.7, from Constants.DEFAULT_SIMILARITY_THRESHOLD, and the parameter is public and documented.

Negative cosine similarity is the second case: '-0.5' sorts above '-0.1', so the worse match ranks higher.

Reproducible example code

Drives HybridRetriever.query with a stubbed vector retriever. No API key and no network.

from camel.retrievers.hybrid_retrival import HybridRetriever


class FakeVR:
    def query(self, query, top_k, similarity_threshold):
        return [
            {"text": "best", "similarity score": "0.9"},
            {"text": "closer", "similarity score": "-0.1"},
            {"text": "further", "similarity score": "-0.5"},
        ]


class FakeBM25:
    def query(self, query, top_k):
        return []


hr = HybridRetriever.__new__(HybridRetriever)
hr.vr = FakeVR()
hr.bm25 = FakeBM25()

print(hr.query(
    query="anything", top_k=3, vector_weight=1.0, bm25_weight=0.0,
    vector_retriever_top_k=50, vector_retriever_similarity_threshold=-1.0,
    bm25_retriever_top_k=50,
)["Retrieved Context"])

Output on 8c791b7:

['best', 'further', 'closer']

Traceback

No traceback. Nothing raises. The ranking is wrong and the call succeeds.

Expected behavior

['best', 'closer', 'further']

Ranked by the numeric value of the score, which is what reverse=True on a similarity is for.

Additional context

auto_retriever.py has the same line inside run_vector_retriever, reached by the same stub route, and produces the same order.

camel/retrievers/cohere_rerank_retriever.py sorts on relevance score, which it stores as a float, so it is unaffected.

Searched: HybridRetriever similarity, hybrid retriever sort, run_vector_retriever similarity, with_score_sorted and similarity score string, against issues and pull requests, both states. The only near hit is #4053, a docstring-formatting pull request for this module, which is closed and unrelated.

A branch with the one-line change in each file, plus a regression test beside each, is ready. Per the AI-Generated Code Policy I am raising this first rather than opening a pull request, and will wait for the issue to be accepted.

AI assistance went into finding and verifying this. Every line of it has had human review and I can defend it.