If you have watched an AI write, you know the ritual.
Tokens appear left to right, one after another, like someone typing very fast.
It feels like proof of intelligence.
It is actually a constraint.
Every mainstream language model, from GPT to Claude to the small model running on your laptop, is locked into a strictly sequential process: emit a token, condition on it, emit the next one.
Never look ahead.
Never go back.
That constraint is now being attacked from an unexpected direction.
This week, two deep explanatory posts are circulating on Hacker News at the same time: a guide from the Kuleshov group at Cornell titled "How to Build a Diffusion Language Model," and Sander Dieleman's post on continuous diffusion language models.
They land on top of a real product wave.
Inception Labs' Mercury generates over 1,000 tokens per second per user on standard GPUs.
NVIDIA's open-weight Nemotron Diffusion models report 2 to 8 times the throughput of comparable autoregressive models while retaining up to 99 percent of their quality.
Google shipped Gemma Diffusion as an open-weights release.
One disclosure before we go further.
I am a backend engineer who runs his own AI agent infrastructure, not an ML researcher.
I have never trained a diffusion model.
Everything below comes from reading the primary sources this week, and I will link them so you can check me.
But I found that the core idea is surprisingly buildable once you see it, and the "which one should I care about" question has a concrete answer now.
That is what this article is for.
Autoregressive generation: the incumbent's superpower and its three defects An autoregressive model generates text the way a very strict typist would.
It predicts the next token given all previous tokens, appends it, and repeats.
This simple recipe won because it is perfectly suited to GPUs during training and produces a clean probability for every token, which makes reinforcement learning post-training straightforward.
But the recipe carries three defects that are structural, not incidental: No error correction.
Once a token is emitted, it is permanent.
Early mistakes compound, because every later token is conditioned on the flawed ones.
Speed is capped by sequence length.
Generating N tokens takes N sequential forward passes.
You cannot parallelize your way out of a process where each step depends on the last one.
Causal attention only.
The model looks backward, never at future context, even when the "future" is text it is about to write anyway.
For years the field accepted these defects as the price of doing business.
Speculative decoding and KV caching shave the cost, but the fundamental loop stayed sequential.
Diffusion language models change the loop itself.
Diffusion for text: start with the whole page wrong, then fix it Diffusion models already generate almost every image you have seen from an AI.
The idea there is beautifully dumb: take a clean image, add a little noise, repeat until you have pure static.
Then train a network to run the tape backward, removing a bit of noise at each step.
Generation means starting from static and denoising your way to a picture.
Text is discrete.
Words are not blurry, and you cannot add a fraction of a word.
So researchers replaced Gaussian noise with something text-shaped: masking.
The most influential formulation, popularized by the Kuleshov group and known as masked diffusion, is best understood as a generative BERT.
You take clean text, hide a random fraction of tokens, and train a bidirectional transformer to fill in the blanks.
Two differences from BERT matter.
The masking rate is randomized across training, which turns out to make the model genuinely generative rather than just a fill-in-the-blanks classifier, and it comes with a principled training objective that closed most of the quality gap with autoregressive models.
Generation then works like this: start from a sequence that is entirely blanks, ask the model to fill in every blank, deliberately re-mask most of the sequence while keeping slightly more tokens fixed than last round, and repeat.
The text assembles itself out of order, wherever the model is most confident.
Here is the sampling loop in pseudocode, so you can hold the whole algorithm in your head: If you know BERT, you already know 80 percent of this.
The remaining 20 percent, the randomized masking schedule and the re-noising loop, is what turns a fill-in-the-blanks model into a generator that can write an entire passage in parallel and revise it mid-stream.
The four upgrades that made it production-ready Plain masked diffusion had real problems: fixed-length output, no way to fix a token after unmasking it, and slow sampling relative to its potential.
The current generation of models stacks four fixes, and the Kuleshov post traces each one.
These are worth knowing by name, because they are the vocabulary the next year of model releases will use.
Block diffusion solves length.
Instead of diffusing one fixed-length canvas, the model generates blocks of tokens conditioned on everything before them, then KV-caches each finished block exactly like an autoregressive model would.
Block size becomes a tuning knob: pick it to match your domain, or to maximize GPU utilization.
Encoder-decoder architectures solve speed.
Researchers noticed diffusion does two jobs, representing finished tokens and denoising broken ones, so modern models split those jobs between a full encoder and a lighter decoder.
Gemma Diffusion and Nemotron Diffusion both use this shape.
Remasking and uniform noise solve error correction.
In remasking samplers, a small subset of already-revealed tokens gets re-masked each step and regenerated, so a grammatical error introduced early can literally be un-written once context arrives.
Uniform state diffusion takes a different route: it replaces tokens with random vocabulary words instead of masks, meaning any token is revisable at any step.
This is what makes parallel generation coherent instead of self-contradictory.
Distillation solves step count.
Progressive distillation, borrowed from image diffusion, trains the model on its own generations to skip steps, halving sampling cost each round.
Combined with the fixes above, this is where the 5 to 10x speedups come from.
There is a fifth layer worth a sentence: post-training.
Diffusion models complicate standard RL because estimating the likelihood of a full sampled sequence is expensive, so techniques like diffu-GRPO approximate it, and newer estimators have already pushed diffusion models to state-of-the-art results on logical and math reasoning benchmarks.
The training recipe that made autoregressive models smart is being ported over.
The model landscape right now So who is actually shipping this?
Four names cover the field today.
LLaDA proved it scales.
An 8B-parameter open-weights masked diffusion model built roughly along the LLaMA recipe, it anchors most academic research in the area.
Mercury from Inception Labs was the first commercial diffusion LLM, and its whole pitch is speed: over 1,000 tokens per second per user on standard GPUs, no exotic hardware.
Its successor claims to rival speed-optimized frontier models at 5 to 10 times their speed.
Gemma Diffusion is Google's open-weights entry, combining the uniform-noise backbone with block diffusion and the encoder-decoder design, and it is already supported in mainstream tooling.
Nemotron Diffusion is NVIDIA's family, scaled up to 35B parameters with a pragmatic twist: a single checkpoint can fall back to plain autoregressive decoding when you want it.
That last detail deserves a pause.
A model that speaks both languages, writing in parallel when you need throughput and sequentially when you want maximum reliability, tells you the industry does not see this as a religious war.
It sees it as a per-request routing decision.
Autoregressive vs diffusion: which one wins, and when Here is the honest comparison, with the limits included, because 99 per