Proposal: EvalPort adapter for Guardrails Validators (portable eval test cases + results)

Hi Guardrails team — I maintain EvalPort (Apache 2.0), an open interchange format for portable LLM evaluation test cases, graders, suites, and results as plain JSON. It's already got 14 standalone framework adapters (AutoGen, CrewAI, Ragas, LangSmith, Braintrust, MLflow, Opik, Arize Phoenix, W&B Weave, UpTrain, Langfuse, Giskard, LlamaIndex, Patronus AI) and one merged upstream integration (Inspect AI).

I've read guardrails/validator_base.py and guardrails/classes/validation/validation_result.py directly and think Validator's shape maps cleanly onto EvalPort's TestCase/Grader/GraderResult:

from guardrails.validator_base import Validator
# Validator.validate(value, metadata) -> ValidationResult
# ValidationResult is one of PassResult() / FailResult(error_message=..., fix_value=...)

That's essentially EvalPort's grader contract already — a pass/fail outcome with an optional explanation and an optional corrected value, applied to one piece of LLM output. Confirmed this actually runs, not just reading the source:

from guardrails.validator_base import Validator, register_validator
from guardrails.classes.validation.validation_result import PassResult, FailResult

@register_validator(name="exact-match", data_type="string")
class ExactMatch(Validator):
    def __init__(self, expected=None, **kwargs):
        super().__init__(**kwargs)
        self.expected = expected

    def _validate(self, value, metadata):
        if value.strip() == (self.expected or "").strip():
            return PassResult()
        return FailResult(error_message=f"Expected {self.expected!r}, got {value!r}")

v = ExactMatch(expected="Paris")
v.validate("Paris", {})   # PassResult(outcome=<Outcome.PASS: 'pass'>, value_override=None)
v.validate("London", {})  # FailResult(outcome=<Outcome.FAIL: 'fail'>, error_message="Expected 'Paris', got 'London'")

Proposed shape, following the same standalone-adapter pattern the other 14 use (adapters/guardrails-openeval-adapter/, to_openeval()/from_openeval(), tests against a real openeval.validate.validate_suite(), not a mock):

• to_openeval(values, validators, ...) — one value per EvalPort test case, {name: Validator} mapping → every validator becomes a grader applied to every test case (mirrors how Guard.use(*validators) runs every attached validator against one output).
• from_openeval(suite) — rebuilds values/metadata_list from an EvalPort suite's input/context/metadata fields, ready to hand to validator.validate(value, metadata) for real.
• batch_result_to_openeval(results, test_case_ids, ...) — converts {validator_name: [ValidationResult, ...]} into an EvalPort ResultSet: PassResult/FailResult → score: 1.0/0.0, passed: True/False, error_message/fix_value preserved under grader_result.metadata.guardrails (nothing dropped, since EvalPort has no native "suggested fix" field).

Two things I'd want your input on before writing code, since the honest answer affects the mapping:

1. Hub-installed validators (guardrails hub install hub://guardrails/...) vs. locally-defined @register_validator classes like the one above — do hub validators expose enough of their identity/config through the base Validator API (without a live hub fetch) to build an honest llm_judge/custom grader distinction, or does that require network access at suite-build time? I'd rather ask than guess and get it wrong.
2. Where would you want this to live — a standalone guardrails-openeval-adapter package (my default, matching every other adapter in the ecosystem), or is there an existing extension-point convention in this repo I should follow instead (I didn't see one in CONTRIBUTING.md, which points new-feature discussion to an issue or Discord, which is what this is)?

Happy to build it either way, tested against the real guardrails-ai package and the real EvalPort validator, same as the other 14. Spec: https://github.com/adhabnr-ux/evalport/blob/main/spec/SPEC.md