This is a Plain English Papers summary of a research paper called BDH-CQ Uses Recurrent Latent Reasoning to Cut ARC-AGI Inference Costs.
If you like these kinds of analyses, you can find more research on AIModels.fyi or follow us on Twitter.
The cost-accuracy trap in visual reasoning Large language models are fundamentally mismatched for visual reasoning tasks.
They're forced to describe every thought out loud, generating token after token to explain their logic.
This verbosity taxes compute budgets, yet paradoxically doesn't improve performance.
Ask a language model to solve an ARC-AGI puzzle (a visual reasoning benchmark designed to test abstract thinking), and it either struggles despite the verbosity or succeeds expensively.
The root problem runs deeper than just inference cost: the model learns from demonstrations by parsing them as language tokens, which is an indirect and inefficient way to absorb a visual pattern.
The efficiency frontier has been unforgiving.
If you want cheap inference, you sacrifice accuracy.
If you want accuracy, you sacrifice cost.
Every model on the leaderboard until recently clustered into one of two camps, and no one had found a path that broke the tradeoff.
BDH-CQ challenges this assumption by proposing something radical: reasoning doesn't need to be visible to work.
The model absorbs demonstrations silently into its internal memory state, then solves problems through private iteration in hidden layers, without generating a single token of intermediate reasoning.
A 150-parameter variant achieves 29.5% pass@2 on the ARC-AGI-1 benchmark at a computed cost of just $0.0007 per task, puncturing through the previous Pareto frontier and establishing a new state of the art in cost efficiency.
Learning through hidden states The core insight is deceptively simple: a model's reasoning process doesn't need to match human communication.
When you learn a new skill from examples, you don't narrate every observation.
You absorb patterns directly into your intuition.
BDH-CQ applies this to neural networks by treating the model's recurrent hidden state as a working memory that continuously absorbs information from demonstrations.
Here's how it actually works.
The model receives a sequence of examples from the demonstration set.
Each example updates its internal state.
By the time the model reaches the query input (the problem to solve), its memory has been shaped by everything it learned from those examples.
It then leverages this primed state to solve the new problem through iterative computation in latent space.
This is fundamentally different from how in-context learning works in language models.
In a transformer, examples appear as tokens in the prompt and the model has to parse them using the same machinery it uses for language understanding.
Here, examples bypass that linguistic bottleneck entirely.
They directly steer the model's latent representation.
The model doesn't need to "read" what it should learn; it can absorb patterns directly.
This reframing solves two problems simultaneously.
First, it's cheaper because the model never generates reasoning tokens.
Second, it might actually learn better from few examples because the information flows directly into working memory rather than being filtered through language parsing.
The approach doesn't fight the architecture; it aligns with what recurrent networks are naturally built to do.
The recurrent mechanism Understanding the architecture requires stepping back to what recurrence actually provides.
A recurrent neural network maintains a hidden state that evolves over time.
At each step, the state updates based on current input while carrying information from all previous steps.
This is the opposite of a transformer, which processes all tokens in parallel.
In BDH-CQ, the hidden state acts as working memory.
When the model processes the first demonstration, its state shifts.
When it processes the second demonstration, the state shifts again, carrying forward