[BFCL] Strict regex-based evaluator gives false negatives even on semantically accurate model responses

Problem

The evaluator misclassifies model's answers that are semantically accurate but formatted differently from the ground truth, as negatives.

Example: model output "8:00 AM" is marked incorrect against ground truth "08:00 AM".

Reproduction

1. Ground truth: "08:00 AM"
2. Model response: "8:00 AM"
3. Evaluation uses strict string equality => flagged incorrect

Expected behavior

Normalize/parse times before comparing, or use a tolerant comparator so "8:00 AM" and "08:00 AM" are treated as equal.

Suggested fixes

• Add a canonicalization step in the evaluator for recognized time formats (accept leading zeros, 12/24h, whitespace, and case differences).
• Prefer parsing into a datetime/time object and comparing that (or format into a canonical string like "HH:MM" or ISO 8601).
• Make the normalization/strictness configurable so projects can opt into tolerant comparisons.

Tests to add

• Unit tests asserting equality for pairs like ("8:00 AM","08:00 AM"), ("8:00","08:00"), ("20:00","08:00 PM").

Severity

Medium — causes false negatives in evaluation metrics.

Notes

If the evaluator already attempts normalization for dates/times, this report may point to a missing format or a bug in the normalizer.

Optional helper: a small canonicalizer

import re
from datetime import datetime

def canonicalize_time(s: str) -> str:
    s = s.strip().upper()
    # Try a simple AM/PM pattern first, allowing optional leading zero
    m = re.match(r'^0?([1-9]|1[0-2]):([0-5][0-9])\s*(AM|PM)$', s)
    if m:
        h = int(m.group(1))
        minute = m.group(2)
        ampm = m.group(3)
        h24 = (h % 12) + (12 if ampm == 'PM' else 0)
        return f"{h24:02d}:{minute}"
    # Fallback: try parsing ISO-like HH:MM
    m2 = re.match(r'^0?([0-1]?\d|2[0-3]):([0-5][0-9])$', s)
    if m2:
        h = int(m2.group(1))
        minute = m2.group(2)
        return f"{h:02d}:{minute}"
    # If parsing fails, return original (or raise)
    return s