AI models are increasingly being asked to design, deploy, secure, and recover production-grade infrastructure — not just write functions that pass a unit test.
That shift changes what a training environment has to be.
It's no longer enough to check that output "looks right." You need a golden reference solution, a deterministic validation suite, and a set of intentionally broken variants that probe exactly where a model's reasoning breaks down under failure.
I've spent the last several months on the other side of this problem — evaluating agentic coding outputs against structured rubrics, designing programmatic verification checks, and writing up edge cases that rubric authors hadn't considered.
Before that, I spent seven years building and debugging the kind of systems these environments are meant to simulate: multi-tenant SaaS platforms with row-level security, AWS infrastructure serving high-traffic e-commerce, and data pipelines processing tens of thousands of concurrent requests.
This post is about where those two things meet — what it actually takes to build an infrastructure RL environment that's reproducible, fair to evaluate, and hard to game.
A golden solution is only as good as its ambiguity budget The first mistake in building any evaluation environment is under-specifying the scenario and over-specifying the solution.
If the task says "deploy a fault-tolerant queue consumer" without pinning down delivery semantics, retry policy, and what "fault-tolerant" means operationally, you'll end up with a golden solution that's just one valid interpretation among several — and you'll penalize a model for a decision the spec never actually made.
This is the same discipline as writing rubrics for coding evaluations: every ambiguous term is a future dispute.
In practice that means, before writing a single line of infrastructure code, defining: The exact failure modes in scope (node loss, network partition, message duplication, clock skew) The invariants that must hold regardless of implementation (at-least-once vs exactly-once delivery, idempotency guarantees, RTO/RPO targets) What counts as "recovered" — not just "the service is up," but that state is consistent Deterministic validation means designing against non-determinism Distributed systems are inherently non-deterministic — that's exactly what makes them hard to evaluate.
A validation suite that works by re-running the same sequence of API calls and diffing output will produce flaky, unfair results the moment retries, timeouts, or eventual consistency are involved.
The pattern that's worked for me, most recently writing a security test suite that validates row-level-security policies against a real Postgres instance rather than mocks, is to validate invariants, not traces: Testing against a real, disposable instance of the actual dependency — a real queue, a real Postgres, a real IAM policy engine — rather than a mock catches the class of bug that mocks are structurally blind to: the ones where the contract you assumed doesn't match the behaviour you get.
I've root-caused production bugs (a minor dependency version bump silently collapsing TypeScript types to , an OAuth token-refresh edge case that only reproduced under real API rate limits) that a mocked test suite would have sailed straight past.
The same principle applies at the infrastructure layer, just with higher stakes per failure.
Defective variants need to fail for the right reason The point of an intentionally broken variant isn't just "does the model notice something is wrong" — it's "does the model correctly diagnose why." A variant with a misconfigured IAM policy that happens to also have a network misconfiguration will teach a model to pattern-match on the wrong signal.
Building these well means treating each defect as a single, isolated hypothesis: One fault per variant.
Resist the urge to combine a broken retry policy and an under-provisioned autoscaling group into one scenario "for efficiency." You'll