fix(scorer): string scorers treat whitespace-only predictions as non-empty
Problem
The three deterministic string scorers guard against an empty prediction, but only against the exact empty string:
# deepeval/scorer/scorer.py
if not prediction:
return 0A whitespace-only prediction (" ", "\t", "\n") is truthy, so it skips
the guard and is compared after normalization. The result is that the same
"no answer" input scores differently depending on whether it contains spaces:
Scorer.exact_match_score("", "") # 0 (empty prediction -> no answer)
Scorer.exact_match_score("", " ") # 1 (whitespace-only -> treated as a real answer)
Scorer.quasi_exact_match_score("", " ") # 1
Scorer.quasi_contains_score([""], " ") # 1For quasi_* this is worse: normalize_text strips whitespace, so a
whitespace-only prediction normalizes to "" and can match an empty target /
empty target list, silently scoring a non-answer as correct.
Proposed change
Treat a whitespace-only prediction the same as an empty one (score 0) in
exact_match_score, quasi_exact_match_score, and quasi_contains_score:
if not prediction or not prediction.strip():
return 0Non-empty predictions are unaffected, so this is backward compatible: real answers keep their exact current scores, and the guard now means "no answer given" instead of "not the literal empty string".
Tests
Adds tests/test_metrics/test_scorer_answer_scoring_hardening.py covering both
"default behaviour unchanged" and the new empty/whitespace consistency for all
three scorers. Offline, no new dependencies.
Source: confident-ai/deepeval