Proposal: deterministic eval for agent action-boundary violations

Describe the feature or improvement

Add an opt-in, deterministic eval for agent action-boundary violations: given a normalized action trace and an explicitly declared sandbox policy, the eval should flag writes outside the allowed roots or destinations, while accepting in-boundary writes and no-write controls.

This targets an execution-boundary failure, not general tool-trajectory drift. It is complementary to #1825: that proposal compares whether a tool sequence or terminal state changed; this proposal checks whether an observed write crossed a declared policy boundary.

Why this is useful

A scalar task score can remain green after an agent writes to an unintended file, directory, or external destination. A small, checked-in trace corpus would make this failure reproducible in CI and turn incident review into a pre-deployment regression check without requiring live network access or real side effects.

Proposed shape

• Use a versioned, secret-free JSON/YAML corpus containing action type, normalized destination, declared boundary, and expected outcome.
• Cover allowed writes, nested allowed paths, traversal/reparse-point attempts, disallowed local roots, external destinations, and a no-write control.
• Report stable per-category counts (true positives, false positives, and boundary misses), with a deterministic overall score and the first violating action.
• Keep the eval opt-in and fixture-only; it must not write to the host, contact external services, or change existing event schemas.

The key maintainer question is whether the supported form should be a model-graded YAML eval over serialized traces, or whether the registry can accept a small deterministic evaluator for this safety boundary. I can prepare the fixture corpus and focused validation once that contract is confirmed.

Validation plan

The regression gate should pass the in-boundary and no-write controls, fail every out-of-boundary category, and report zero false positives on the known-good traces. The fixture set should contain at least 15 high-quality cases and avoid user data or secrets.