#3220·deepeval

fix(metrics): whitespace-only actual_output/expected_output pass the empty-param guard

Author: Asthenia0412Created Aug 31, 2026Updated Sep 13, 2026

Problem

check_llm_test_case_params rejects a required actual_output only when it is the empty string (== ""). A whitespace-only value (" ", "\n\t", …) passes the guard, and deterministic metrics then .strip() it before comparing:

  • ExactMatchMetric with actual_output=" " and expected_output=" " strips both to "" and scores 1.0 — a silent false pass for two effectively-empty strings.
  • PatternMatchMetric similarly strips before matching.

This is also inconsistent with the scoring layer: the string scorers already treat whitespace-only predictions as empty (Scorer.exact_match_score / quasi_exact_match_score / quasi_contains_score return 0 for them). The metric-param guard should enforce the same convention so a degenerate test case is surfaced as a missing parameter rather than silently passing.

Proposed change

In check_llm_test_case_params, treat a whitespace-only string (str.strip() == "") the same as an empty string for the required actual_output (and symmetrically expected_output, which is currently not validated at all). The check remains isinstance(..., str)-guarded so non-string (multimodal) outputs are unaffected.

Backward compatible: only rejects values that were previously degenerate. No new dependencies; fully offline-testable.