[Bug]: KLDivergence(smoothing=0.0) raises ZeroDivisionError when a token is missing from the other text
What component(s) are affected?
- Opik Python SDK
- Opik Typescript SDK
- Opik Agent Optimizer SDK
- Opik UI
- Opik Server
- Documentation
Opik version
- Opik version: current
mainatc610bf10
Describe the problem
KLDivergence accepts smoothing=0.0 (the constructor clamps with max(0.0, smoothing) and _smooth treats 0.0 as "no smoothing"), but with no smoothing the divergence is undefined whenever a token of one text is absent from the other, and _kl in sdks/python/src/opik/evaluation/metrics/heuristics/distribution_metrics.py crashes with a bare ZeroDivisionError from p_val * math.log(p_val / q_val).
Every other unscorable input in this metric (empty candidate, empty reference, empty tokenization) is reported as a MetricComputationError, which is what the evaluation engine expects a metric to raise, so this case should be reported the same way with a message that points at the smoothing option.
Reproduction steps and code snippets
from opik.evaluation.metrics import KLDivergence
KLDivergence(smoothing=0.0, track=False).score(output="cat dog", reference="cat")Error logs or stack trace
File ".../opik/evaluation/metrics/heuristics/distribution_metrics.py", line 342, in _kl
divergence += p_val * math.log(p_val / q_val)
~~~~~~^~~~~~~
ZeroDivisionError: float division by zeroHealthcheck results
Not applicable: local metric computation, no backend involved.
Proposed fix: raise MetricComputationError from _kl when the smoothed probability of a token is zero, with a message naming the token and suggesting a positive smoothing. I have a fix with regression tests and will open a PR. This report was prepared with AI assistance (Claude Code); I reproduced it locally against current main.
Source: comet-ml/opik