Every failure I am about to describe produced a clean run.
No exception, no stack trace, no red build.
Each one produced a plausible number that I believed for longer than I should have.
That is the category of bug I have come to fear most.
A crash tells you it crashed.
A silently broken dataset tells you nothing at all, and your metrics will politely agree with it.
Here are three from the last year, all from my own work, all found late.
1.
The dataset that was 92% one category I had a training set of 688 records for a multi-category vision-language task.
Thirteen categories.
Reasonable size for a fine-tune, already used in a completed training run whose results I had written up.
While preparing a stratified split, I joined the records back against the source annotations and actually counted the categories. 630 of 688 were a single category: scene captions.
Zero examples of traffic signals.
Zero of planning.
Zero of uncertainty.
Several categories the evaluation explicitly measured had no representation in training at all.
The previous fine-tune had shown gains on some of those very categories.
I had interpreted this as the model learning the task.
The real explanation was duller and more useful: the model had learned the answer format from caption supervision, and format alignment alone was enough to move a multiple-choice score.
Nothing category-specific had been learned, because nothing category-specific had been shown.
The root cause was upstream and boring.
The conversion script I inherited only rewrote file paths and dropped records with missing frames.
It faithfully preserved a caption-only selection made further up the chain.
It had no opinion about balance because nobody had asked it to have one.
What I changed: the composition of a training set is now an artifact I generate and inspect before any run, not a property I assume.
A category histogram takes seconds.
I had not looked, for months.
2.
The 18-hour run that converged perfectly to nothing Large model, QLoRA, multi-GPU, 8 camera views per sample.
Roughly a full day of compute.
The loss curve was beautiful.
It fell from 19.4 to 15.7 over the first 26 steps, then to 0.078 by step 51, then flattened near 0.02 and stayed there.
Token accuracy reached 0.99.
Gradient norms decayed smoothly.
Nothing in the training telemetry looked wrong.
The task metric was 0.10.
The cause: I was supervising on free-text answers, median 11 words, phrased like "One should keep to the right side of the road and drive slowly." The evaluation was four-option multiple choice, scored on the index of the chosen option.
The model learned, quickly and correctly, to reproduce the style of the training answers.
That is genuinely what the loss was asking for, and it maxed it out in about 50 steps.
It was never asked to select an option, so it never learned to.
Training and evaluation were measuring different tasks.
Both were internally consistent.
Neither could detect the other's disagreement.
What I changed: before a long run starts, I now write down what the eval measures and what the loss optimizes, in one line each, and check they describe the same thing.
When loss collapses to near zero within a few dozen steps, I treat that as an alarm rather than a success.
Genuine learning on a hard task does not look like that.
3.
The dataset that was 40% smaller than its file size claimed Assessing a large public robotics corpus for adoption.
The headline: 3.91 TB compressed, 4.65 TB extracted, hundreds of shards, dozens of buildings, thousands of hours of teleoperated manipulation.
The number that mattered was buried in the paper's method section.
Only about 60% of the raw data converted into reliable 3D flow annotations.
The rest failed depth estimation, camera pose optimization, or point tracking.
Then a further filter kept only trajectories with actual robot-object contact and real object motion.
Nominal size: thousands of hours.
Training-ready size after both filters: roughly 500 hours.
Every plan