Research

The founding programs of AI were engineered reasoners: explicit states, explicit operators, a difference list you could print and read. Newell and Simon’s General Problem Solver was running means-ends analysis by the late 1950s — every step inspectable by design. The statistical turn came decades later. In the history of artificial reasoning, next-token prediction is the newcomer, not the default.

A natural hope for language models is that they can catch their own mistakes: generate an answer, critique it, revise. A 2023 study from Google DeepMind tested that hope directly and found it wanting — without external feedback, self-correction did not improve reasoning and often made it worse. Reviewers of human work have known the underlying principle for a long time.

In 1983, cognitive scientist Dedre Gentner formalized what separates a good analogy from a seductive one: good analogies map relations between things; bad ones match appearances. Every 'we had one just like this' in a case file is a bet on that distinction — and it is the exact distinction next-token prediction is worst at.

In 2023, OpenAI researchers compared two ways of training a model to recognize good reasoning: reward the right answer, or reward each right step. Step-level supervision won decisively. Regulators will find the result unsurprising — it is how they have evaluated human decision-making all along.

The fastest way to run a big model turns out to be letting something small do most of the talking. Speculative decoding drafts tokens cheaply and verifies them in parallel — provably without changing the output. DeepSeek-V3 builds the drafter into the model itself and gets ~1.8× decoding speed. The pattern — cheap generation under strict verification — is one we recognize. The guarantee that makes it free at the token level is exactly what verification loses one level up, and that difference is worth understanding precisely.

A reasoning model carries two kinds of momentum. One is worth keeping — the capability encoded in its weights by training. The other is the reason it can argue itself out of the right answer: the pull to stay consistent with whatever it said first. Chain-of-thought struggles to interrupt the second, because the check is written by the same running generation. A separate call can reduce it — and that, we argue, is most of why IRG is a different thing than a prompt.

VibeThinker-3B scores 94.3 on AIME26 with three billion parameters, matching models orders of magnitude larger. The claim underneath the benchmark is the interesting part: reasoning compresses aggressively while knowledge does not — meaning reasoning is a separable artifact. That has consequences for how production systems should be built.

For years the industry pulled three levers: more parameters, more data, more test-time compute. Memory stayed entangled in the weights. DeepSeek’s Engram pulls a fourth lever — explicit, conditional memory — and the early numbers suggest it was underexplored for no good reason. The deeper signal is architectural: the monolith is unbundling.