Don't Golden-File an Agent Patch. Golden-File the Relation.

A recorded expected value is a leak. An agent that can read can patch until that line is green and leave every unlisted input broken. A metamorphic relation does not publish . It only publishes a constraint the output must keep under a known transform. That is the gate worth automating. Fixtures still matter, but only as seeds. Flaky tests still need a freeze, but the freeze must not cover the relation itself. This article is a proposed layout, not a production case study. No runtime metrics are claimed. The commands and modules below are labeled so they can be copied into a scratch repo and executed against your own function under test. Why snapshots fail as a merge gate Golden files encode one transcript. An agent patch is a search over many transcripts. If the search can see the answer key, the cheapest passing program is a lookup table for the keys in tree. That program is green. It is also wrong on the next customer file. Property-style checks reduce that leak because they do not ship the answer. They still need a seed corpus, a replay runner that the patch cannot edit, and a quarantine file that expires. Mix those three and you get a gate that fails closed when the agent rewrites tests, when a fixture drifts, or when a flake is used to hide a broken invariant. Three relation classes worth encoding first Start with relations you can state in one line. If you cannot state the line, you do not have a gate. You have a recorder. Idempotence. for normalizers, formatters, and canonicalizers. Round-trip. equals on the fields you actually guarantee, not on whitespace you do not. Oracle-free comparison. relates to for a transform you control: shuffle independent rows, rename equivalent keys, NFC vs NFD unicode, scale a quantity and its unit together. These are not universal laws. They are hypotheses about your function. Write them down as code. Keep the seed inputs boring. The relation, not the seed, does the work. Seed fixtures are not expected outputs A seed is an input the runner is allowed to read. It is not a blessed byte dump of . Store seeds as files with no sibling. If a reviewer adds an expected file next to a seed, the gate should fail the build. That single rule stops the suite from turning back into a recorder. Proposed tree: The agent may write . It must not write . Enforce that in CI with a path check, not with a comment. Artifact: a replay runner the patch cannot satisfy by editing tests The module under test is a CSV amount normalizer. Treat the following as a proposed example, not as measured production code. Relation tests consume seeds and never mention a golden output string. should refuse two things: an empty seed directory, and any patch that touches . If your agent workflow does not produce a git diff against , replace the subprocess with a permission check: the verify job runs as a user that has read-only access to and read-write access only to a scratch directory. Freeze flakes. Do not freeze relations. Network calls, clock reads, and unordered log lines flake. Idempotence of a pure normalizer does not. A quarantine file is useful. It is also the easiest place to hide a regression if you let it list relation test ids. Proposed schema: Loader rules, proposed and strict: A freeze without an expiry is a deleted test. CI should treat an expired row as a failing test, not as a skip. Re-open the row or fix the environment. Do not extend from the agent workspace. Numbered workflow Run generation and verification as two jobs with two disks. The patch is a candidate. The relation suite is the judge. Lock the oracle. Copy from onto the verifier. Mark it read-only. Do not check the agent's worktree out on top of it. Seed, do not record. Add the smallest CSV files that exercise headers, UTF-8, empty amounts, and two-row permutation. Commit them on before the agent runs. Generate the patch off the verifier. Point the coding agent at a checkout that contains and a task prompt, not . Apply the patch to a throwaway tree.