#8337·opik

[Bug]: KLDivergence(smoothing=0.0) raises ZeroDivisionError when a token is missing from the other text

Author: anishmehta24Created Sep 15, 2026Updated Sep 15, 2026

What component(s) are affected?

  • Opik Python SDK
  • Opik Typescript SDK
  • Opik Agent Optimizer SDK
  • Opik UI
  • Opik Server
  • Documentation

Opik version

  • Opik version: current main at c610bf10

Describe the problem

KLDivergence accepts smoothing=0.0 (the constructor clamps with max(0.0, smoothing) and _smooth treats 0.0 as "no smoothing"), but with no smoothing the divergence is undefined whenever a token of one text is absent from the other, and _kl in sdks/python/src/opik/evaluation/metrics/heuristics/distribution_metrics.py crashes with a bare ZeroDivisionError from p_val * math.log(p_val / q_val).

Every other unscorable input in this metric (empty candidate, empty reference, empty tokenization) is reported as a MetricComputationError, which is what the evaluation engine expects a metric to raise, so this case should be reported the same way with a message that points at the smoothing option.

Reproduction steps and code snippets

python
from opik.evaluation.metrics import KLDivergence

KLDivergence(smoothing=0.0, track=False).score(output="cat dog", reference="cat")

Error logs or stack trace

  File ".../opik/evaluation/metrics/heuristics/distribution_metrics.py", line 342, in _kl
    divergence += p_val * math.log(p_val / q_val)
                                   ~~~~~~^~~~~~~
ZeroDivisionError: float division by zero

Healthcheck results

Not applicable: local metric computation, no backend involved.


Proposed fix: raise MetricComputationError from _kl when the smoothed probability of a token is zero, with a message naming the token and suggesting a positive smoothing. I have a fix with regression tests and will open a PR. This report was prepared with AI assistance (Claude Code); I reproduced it locally against current main.