EvalPort adapter for `rasa test nlu` results (intent_report.json / intent_errors.json)

EvalPort (https://github.com/adhabnr-ux/evalport) is an open interchange spec + SDK for portable LLM/NLU evaluation datasets and results, so eval output isn't locked to one framework's report format.

rasa test nlu already produces a genuinely portable-shaped result. In rasa/nlu/test.py, evaluate_intents() builds its predictions list straight from IntentEvaluationResult namedtuples (intent_target, intent_prediction, message, confidence) — one entry per utterance, essentially a test-case-level result already. That's the same data that gets written to disk as intent_errors.json/intent_successes.json via _get_intent_errors/write_intent_successes (each item {"text", "intent", "intent_prediction": {"name", "confidence"}}), while the per-label sklearn classification_report (precision/recall/f1/support per intent) is dumped separately as intent_report.json via _dump_report. run_evaluation() in the same module returns all three legs — intent_evaluation, entity_evaluation, response_selection_evaluation — as one dict, so there's a single natural place to hook a converter.

Before (results/intent_errors.json after rasa test nlu --errors):

[
  {"text": "i wanna cancel my order", "intent": "cancel_order",
   "intent_prediction": {"name": "check_order_status", "confidence": 0.61}}
]

After, with a rasa-openeval-adapter reading the dict run_evaluation() returns (or the on-disk intent_report.json / intent_successes.json / intent_errors.json trio):

from rasa_openeval_adapter import results_to_openeval
from openeval.validate import validate_result_set

result_set = results_to_openeval(intent_evaluation_dict, suite_id="rasa-nlu-regression")
assert validate_result_set(result_set).valid

Each IntentEvaluationResult becomes one EvalPort test-case result (actual_output=intent_prediction, expected_output=intent_target, pass/fail from equality, score=confidence); the sklearn report dict becomes suite-level per-intent aggregate metrics. CVEvaluationResult(train, test, evaluation) from perform_nlu_cross_validation's cross-validation path maps just as cleanly to multiple named ResultSets (one per fold split).

I'd like to build this as adapters/rasa-openeval-adapter/ in the EvalPort repo — same shape as the other adapters there (pyproject.toml depending on evalport-sdk, to_openeval()/from_openeval()/results_to_openeval(), tests against the real validator, a README). It would work entirely from the outside against rasa test's output, so no changes to this repo are needed — flagging here first per the contributing guide, in case there's a preferred hook point (in-memory dict vs. the on-disk JSON files) or something similar already underway.

https://github.com/adhabnr-ux/evalport

Posting as an independent contributor / maintainer of EvalPort, not affiliated with Rasa.