Originally published on hexisteme notes.
I run pre-registered checks against a live system, read the verdict, and move on — that's the whole point of pre-registering them, so I don't get to argue with the result after the fact.
Most of the time the discipline pays for itself.
This time it passed, and the pass was wrong, and the reason it was wrong is more interesting than the failure itself: the check could not have returned anything else, whatever had actually happened to the file under test.
The question I was probing something narrow: does a hand-made audio crossfade survive a round trip through DaVinci Resolve?
Build a timeline with a crossfade sitting on a cut, export it to FCPXML 1.10, re-import it, and see whether the crossfade is still there.
Third-party documentation says transitions are invisible to and unmodifiable by the scripting API.
Believing that, I pre-registered a judgment method that never looks at timeline structure at all: render audio around the splice and classify it by waveform shape.
The judge, exactly as pre-registered: render two seconds either side of the cut, downsample to 8 kHz mono, compute a 20 ms sliding-window RMS envelope — 202 windows across the render — and take the largest normalized step between adjacent windows.
Above 0.5, call it a hard cut: the fade is gone.
Below 0.5, call it a gradual ramp: the fade survived.
The probe came back pass — gradual ramp, max step 0.4761, under the 0.5 threshold.
Exit 0, all green.
The crossfade had actually been lost at the export step.
The pass was a false confirm, and I only found that out by going back in with a second, read-only inspection after the fact.
Why the check could not fail The prep instructions for this probe — which I also wrote — said the easiest way to get two adjacent audio items with enough handle to build a crossfade is to take one continuous clip and blade-split it in the middle.
That's a completely reasonable instruction on its own.
A crossfade needs overlap media on both sides of the cut, and splitting a continuous take is the cheapest way to guarantee that.
It also quietly destroys the judge.
If both sides of the boundary come from the same continuous recording, then losing the crossfade doesn't produce a hard edge in the waveform — it just reconnects the same continuous audio it started as.
Render across that junction and you get a smooth signal whether the fade survived or not.
Gradual ramp either way.
The check was going to say pass regardless of the real answer, because the one input feature it needed in order to discriminate — a genuine discontinuity at the boundary — was never present to begin with.
The judge's implicit precondition was never written down anywhere in the pre-registration: the content on the two sides of the boundary has to actually differ, or a hard cut has nothing to show up as.
Nobody checked that precondition against the prep procedure, because the same person designed both, and designing both feels like it should make them consistent by construction.
It doesn't.
A metric and the data-generating procedure that feeds it are two separate design decisions, and their interaction needs to be checked on its own.
Two flags already sitting in the passing run The part that stings is that this wasn't hidden.
Two pieces of evidence that the pass was hollow were already sitting in the same results file the pass came from.
First, where the largest step happened.
The junction was at 2.00 s into the render.
The maximum step was at 0.38 s — nowhere near it.
None of the top five steps in the whole window fell within ±0.25 s of the junction.
The metric wasn't measuring the cut at all; it was picking up ordinary dynamics in the audio elsewhere in the clip.
Second, a keyword scan of the exported XML for anything fade- or transition-related came back with zero hits.
Neither of these shows up if you only read the top-level verdict. is one bit.
The evidence that the bit was meaningless was sitting a few fields deeper in the sam