Proposal: agent-workflow replayability conformance fixture (clean-run hashes, interrupted tool-call recovery, tampered artifact, field-loss report)

Following up from the community Slack #clearml-dev thread with Martin.B, who suggested GitHub issues as the right place for code and concrete suggestions. This is a proposal to contribute a narrow, test-only conformance fixture, not a product pitch.

The case

One tool-calling AI agent workflow (an AI agent, not clearml-agent) tracked as a ClearML task: it pulls a small dataset, runs an evaluation tool, and writes a report artifact. The fixture records and replays four things:

1. Clean run: expected task and artifact IDs plus content hashes recorded as the baseline. Re-running reproduces the same hashes or the fixture says exactly which field drifted.
2. Interrupted tool call: the same run killed between the tool's side effect (artifact committed) and task completion. Recovery should replay the committed artifact rather than recompute it, and the fixture pins whichever behavior ClearML's task records actually produce today, so a future change is a deliberate expectation flip rather than silent drift.
3. Tampered artifact: the same run with one artifact byte flipped. Verification should fail closed on the content-hash mismatch, never report the run as reproduced.
4. Field-loss report: a passing test that states, as data, what the task records cannot represent about the interrupted case (for example, whether a committed artifact of an unfinished task is distinguishable from an uncommitted one on resume). The absence itself is the finding.

Prior art

I shipped the same shape last week as a checkpoint-recovery conformance fixture for the open-multi-agent project: https://github.com/open-multi-agent/open-multi-agent/pull/452 (four recovery cases plus a field-loss report, pinned against current checkpoint granularity). The ClearML fixture would follow that proven pattern using only repository-native test conventions.

Scope and questions

• Test-only: no production code changes, no new dependencies.
• Where should it live (tests/, examples/, or somewhere else), and is there a preferred schema for expected-run records?
• No real credentials or external services: mock or local server, whatever the repo prefers.

Process note: I use my own fair-source tools (Forum, Gather) to generate private evidence while developing fixtures, but nothing in the contributed fixture would depend on them. Developed with AI assistance, author-directed and disclosed, as with all my public work.