#3186·deepeval

fix(scorer): string scorers treat whitespace-only predictions as non-empty

Author: Asthenia0412Created Aug 31, 2026Updated Sep 8, 2026

Problem

The three deterministic string scorers guard against an empty prediction, but only against the exact empty string:

python
# deepeval/scorer/scorer.py
if not prediction:
    return 0

A whitespace-only prediction (" ", "\t", "\n") is truthy, so it skips the guard and is compared after normalization. The result is that the same "no answer" input scores differently depending on whether it contains spaces:

python
Scorer.exact_match_score("", "")     # 0  (empty prediction -> no answer)
Scorer.exact_match_score("", "   ")  # 1  (whitespace-only -> treated as a real answer)

Scorer.quasi_exact_match_score("", "   ")   # 1
Scorer.quasi_contains_score([""], "   ")    # 1

For quasi_* this is worse: normalize_text strips whitespace, so a whitespace-only prediction normalizes to "" and can match an empty target / empty target list, silently scoring a non-answer as correct.

Proposed change

Treat a whitespace-only prediction the same as an empty one (score 0) in exact_match_score, quasi_exact_match_score, and quasi_contains_score:

python
if not prediction or not prediction.strip():
    return 0

Non-empty predictions are unaffected, so this is backward compatible: real answers keep their exact current scores, and the guard now means "no answer given" instead of "not the literal empty string".

Tests

Adds tests/test_metrics/test_scorer_answer_scoring_hardening.py covering both "default behaviour unchanged" and the new empty/whitespace consistency for all three scorers. Offline, no new dependencies.