On July 22, 2026, White House OSTP director Michael Kratsios accused Moonshot AI of distilling Anthropic's Fable 5 to build Kimi K3, and Treasury put sanctions on the table.
No technical evidence was published, and researchers immediately pointed out the timeline problem: by K3's launch day, Fable 5 had been publicly reachable for roughly 18 days in total.
Everyone argued about whether it happened.
Almost nobody asked the more useful engineering question: if it did happen, what would Moonshot actually have received?
Here is the experiment that answers it.
In 2025, the Berkeley team behind Sky-T1 fine-tuned a model on long reasoning traces whose final answers were wrong.
Accuracy dropped by 3.2 points.
They randomized half the numbers inside the reasoning steps: 3.3 points.
Then they shuffled the order of the steps, and performance collapsed.
The content of a distill is nearly disposable.
The structure is the payload.
That is the folk model's blind spot.
The folk model says: pour a strong model's outputs into an open model, get a model at the strong model's level.
What actually crosses the wire is mostly the shape of the reasoning, and the size of the gain is set by your base model, not by how smart your teacher was.
TL;DR A "distill" is a dataset of teacher traces, not a set of weights.
Fine-tuning on it reliably transfers output structure (long chain-of-thought, backtracking, blocks) and only narrowly transfers capability.
Evidence: models trained on traces with wrong answers lose about 3.2 points versus correct ones, and randomizing half the numbers in the traces costs 3.3 points on AIME
2024.
Real capability gains do exist (DeepSeek's R1 distills beat RL on the same base by 25 points on AIME), but they cost 800k rejection-sampled samples and a full fine-tune, not 8k samples and a LoRA.
The headline benchmark jumps you see on Hugging Face are very often eval artifacts.
What a "distill" actually is When someone says "I poured a distill into Qwen," they are not moving weights.
They are running SFT on a dataset of teacher outputs.
There are two distinct channels, and they behave differently: Black-box (traces) White-box (logits) What you collect Generated text, usually prompt + long CoT + answer Full next-token distributions, or hidden states Loss Cross-entropy on the teacher's tokens KL between teacher and student distributions Bits per token One sampled token The whole distribution ("dark knowledge") Needs API access Weights, and enough GPUs to run the teacher Works across model families Yes Yes, if tokenizers align Anything distilled from a closed API (Claude, GPT, Gemini) is black-box by construction.
You cannot get logits out of an inference endpoint.
This matters more than it sounds: black-box distillation is the low-bandwidth channel, and it is the only one available in the scenario the White House described.
The whole current ecosystem is downstream of two decisions made four months apart.
In September 2024, OpenAI hid o1's raw chain of thought and explicitly listed competitive advantage among the reasons.
In January 2025, DeepSeek shipped R1 with full traces exposed under a permissive license, and within weeks the distill wave existed: s1, LIMO, Sky-T1, and a Hugging Face shelf of "-Distill" repos.
Traces are the substrate; whoever exposes them feeds the ecosystem, and hiding them is anti-distillation policy by another name.
None of this is exotic in-house either: Google's own Gemini 1.5 technical report states that Flash is online-distilled from the much larger Pro.
Every lab does this to its own models.
The fight is only ever about doing it to someone else's.
Kimi is the interesting inverse case.
K3 shipped as open weights in late July 2026 at 2.8 trillion parameters, so white-box distillation from Kimi is legally and technically on the table.
The gate is not access, it is the inference bill for generating traces from a model that needs roughly 1.4 TB of fast memory resident before you load any context.
The case that it genuinely works DeepSeek ran the cleanest public experiment on this, and it is still the strongest pro-distillation datapoint we have.
They generated about 600k rejection-sampled reasoning traces plus 200k general samples from R1, then ran plain SFT for two epochs on off-the-shelf open bases.
No RL on the students at all.
Then they asked the obvious control question: what if you skip the teacher and just run large-scale RL on the same base?
Qwen-32B base, three treatments AIME 2024 MATH-500 GPQA-D LiveCodeBench QwQ-32B-Preview (reference) 50.0 90.6 54.5 41.9 RL directly on the base, 10k+ steps 47.0 91.6 55.0 40.2 SFT on 800k R1 traces 72.6 94.3 62.1 57.2 That is a 25 point gap on AIME in favor of distillation, against an RL run that cost far more compute.
DeepSeek's own conclusion was blunt: distilling a powerful model into a smaller one works, while small models relying on large-scale RL need enormous compute and may still lose.
This is not just curve-sharpening either.
A widely cited ICML/NeurIPS 2025 analysis measured pass@k rather than pass@1 and found the two methods differ in kind.
RLVR raises pass@1 while narrowing the reasoning boundary at large k, because the paths it reinforces were already in the base's sampling distribution.
Distillation raises the curve at every k (on Qwen-7B: pass@1 from 28% to 45%, still above 90% at pass@256), meaning genuinely new reasoning patterns entered the model.
So: yes, capability moves.
Hold that thought.
The case that it isn't what you think Back to the corruption experiment from the intro, with the setup spelled out.
The Sky-T1 team first got +40 points on AIME 2024 by fine-tuning Qwen2.5-32B-Instruct on just 17k long-CoT traces distilled from R1.
Only then did they start breaking the training data on purpose, and found that only structural damage (shuffling, inserting, deleting steps) actually hurt, while wrong answers and randomized numbers cost ~3 points each.
Their conclusion sits in the paper's own title: structure, not content, is what matters.
Two more datapoints point the same way. s1 hit strong reasoning numbers with 1,000 samples.
LIMO used
817.
If a thousand examples move a benchmark 40 points, you are not transferring a frontier lab's knowledge in a thousand examples.
You are flipping a switch that was already wired.
This is the same finding that killed the first imitation wave in
2023.
Berkeley's "False Promise of Imitating Proprietary LLMs" found crowd workers rated ChatGPT imitators as competitive, while targeted benchmarks showed they closed little to none of the gap: they mimicked style, not factuality.
Thinking Machines said the same thing in 2025 about off-policy distillation, that the student learns the teacher's style and confidence without necessarily learning its accuracy.
A worked example at 0.01% of the weights Here is the exact thing the question is usually about, done in public and documented honestly.
Someone took , generated ~7.8k reasoning traces from Kimi K2.6 via OpenRouter, and ran SFT with Unsloth and LoRA.
Attention-only adapters, , 980 steps, about 21 hours on a single H200.
Trainable parameters: 3.44M out of 35.1B.
That is 0.01% of the model.
The model card then does something almost nobody does.
It reports evals that undercut the model: Benchmark, same pipeline both sides Base Qwen3.6-35B-A3B Kimi-distill MATH-500 (0-shot, ) 53.0 47.0 GPQA Diamond (0-shot CoT) 79.29 75.25 GSM8K (8-shot, strict-match) 64.0 92.67 The author's read on that GSM8K number is the important part: the base scoring 64% is implausible for a frontier 35B-A3B, and the likely cause is that the few-shot template never triggers the base's thinking mode.
So the +28.67 point "win" measures "my pipeline rewards models that always think," not capability.
His stated conclusion is that the run provides no evidence the distillation improved raw reasoning over the base.
What it does provide is a guarantee: the distill emits blocks regardless of prompt shape, where the base's thinkin