#1819·visdom

Experiment search ranks a text value above the highest number when sorting descending

Author: piyush182004Created Sep 10, 2026Updated Sep 10, 2026

Bug

FIX:- #1818

ExperimentStore.search(sort_by=..., descending=True) ranks a run whose sort value is text above the run holding the highest number.

A field can hold a number on some runs and text on others — a crashed run logged as "failed", a pending one as "n/a". _order_key handles that by tagging numbers (0, ...) and everything else (1, ...) so numbers group first. But _rank_key returns that tag inside the sort key and _rank applies reverse=descending to the whole key, so the tag reverses too and the text group jumps ahead of every number.

Reproduce

python
store.log_experiment("baseline", params={"accuracy": 0.72})
store.log_experiment("tuned",    params={"accuracy": 0.91})
store.log_experiment("crashed",  params={"accuracy": "failed"})

store.search(sort_by="accuracy", descending=True)

Actual:

['crashed', 'tuned', 'baseline']

Expected:

['tuned', 'baseline', 'crashed']

The best model is listed second, behind a run that never produced a number.

Worse with paging

_rank's heap path (heapq.nlargest) selects on the same key, so bounded queries drop real results:

python
store.search_page(sort_by="accuracy", descending=True, limit=2)
# → ['crashed', 'tuned']   — 'baseline' evicted for a run with no score

Notes

Ascending order is correct; only descending is affected. No error or warning is raised, which is likely why it has gone unnoticed.

Reproduces on dev @ 690da11. It reduces to plain Python tuple comparison with no visdom involved, so it is not environment-specific:

python
def order_key(v):
    return (0, float(v), '') if isinstance(v, (int, float)) and not isinstance(v, bool) else (1, 0.0, str(v))

rows = [(5, 'run_five'), (10, 'run_ten'), ('pending', 'run_str')]
sorted(rows, key=lambda r: order_key(r[0]), reverse=True)
# → [('pending', 'run_str'), (10, 'run_ten'), (5, 'run_five')]

Source: py/visdom/experiments/store.py, _rank_key (~line 119) and _rank (~line 132).