[SDK] Distribution metric docstrings advertise worked examples the metrics never produce

Summary

Three worked examples in sdks/python/src/opik/evaluation/metrics/heuristics/distribution_metrics.py print values that the metrics never produce. Measured on main at dbda1ff4d, Python 3.11, scipy 1.17.1:

Reproduce with:

from opik.evaluation.metrics import JSDivergence, JSDistance, KLDivergence

print(JSDivergence(track=False).score(output="cat cat sat", reference="cat sat on mat").value)
print(JSDistance(track=False).score("a a b", reference="a b b").value)
print(KLDivergence(track=False, direction="avg").score("hello hello world", reference="hello world").value)
# 0.6748107189619399
# 0.08170416594551039
# 0.05776219000072439

Why nothing catches it

Every example in this file is marked # doctest: +SKIP, and pytest is not configured with --doctest-modules (no addopts in sdks/python/pyproject.toml adds it), so the numbers are only ever read — never executed. The same pattern covers 19 example sites across evaluation/metrics, but I am reporting only the ones I could execute without optional model dependencies or credentials, which are these three.

This is not a rounding or a library-version artifact. Recomputing the same two quantities with nothing but math and collections.Counter:

JSD("cat cat sat", "cat sat on mat") = 0.325189  ->  1 - 0.325189 = 0.674811
JSD("a a b", "a b b")                = 0.081704
KL avg, natural log                   = 0.057762   (the same quantity in bits would be 0.083333, which is also not 0.0583)

…matches the shipped code to the last digit. 0.188 is exactly 1 - 0.812, i.e. it was carried across from the first example's arithmetic instead of being measured from the call it sits under — so the two JSD numbers are consistent with each other and inconsistent with the code, which is the shape that survives review by reading alone.

Impact

help(JSDivergence) and IDE tooltips are the only place these numbers appear, and they are the fastest way to learn what the metric returns. A developer using the documented example to sanity-check their own pipeline — or to set a similarity threshold — will see 0.675 where the docs promise 0.812, and the natural conclusion is that their tokenizer, their scipy, or their usage is wrong. For JSDivergence specifically, the advertised value sits on the opposite side of the common 0.7 / 0.8 threshold guesses people make for "similar enough".

Options

• (a) Correct the three values and add a unit test that reads the number out of each docstring and asserts the metric produces it. This is what I have in a branch and can open as a PR immediately; the test makes the drift structurally impossible for the values, though the example inputs are duplicated in the test rather than parsed from the docstring.
• (b) Same as (a), plus drop # doctest: +SKIP for these three examples and run them via --doctest-modules limited to evaluation/metrics/heuristics/distribution_metrics.py in the SDK test job. That also pins the inputs, at the cost of a CI-scope decision that is yours to make.
• (c) Just delete the three numeric lines and keep the examples as usage, not as expected output.

No claim of a security issue here, and no behaviour change is involved — the metrics themselves compute the right thing; only the documentation is out of step.