Grade the Steps, Not the Answer: What Process Supervision Shows
This is the first in a weekly series reading one research paper closely, in plain language, for compliance and supervisory readers. Up first: Lightman et al., “Let’s Verify Step by Step” (OpenAI, 2023). The researchers trained two “graders” of model reasoning: one rewarded only correct final answers, the other rewarded each correct step. On hard math problems, the step-level grader was decisively more reliable at picking correct solutions — and it caught a failure mode the answer-level grader systematically rewards: right answers reached by wrong reasoning. The supervisory translation is direct: outcome testing alone cannot distinguish a sound decision process from a lucky one, and the step—not the answer—is the natural unit of oversight.
The paper
A language model working a hard problem produces a chain of steps and a final answer. If you want a second model to judge that work — to pick the best of many attempts, or to flag bad ones — you must decide what to train the judge on. Outcome supervision labels whole solutions by whether the final answer is correct. Process supervision labels every individual step. The paper builds both judges from the same base model and compares them on MATH, a benchmark of competition mathematics problems, releasing 800,000 human step-level labels (PRM800K) along the way.
The result was not close. Selecting among many attempted solutions, the process-supervised judge solved 78% of problems from a representative test subset — reliably ahead of the outcome-supervised judge, with the gap widening as more attempts were considered. More interesting than the score is why: an outcome judge must credit every path that lands on the right number, including paths that get there through cancelling errors, unjustified leaps, or outright false statements. It is structurally blind to the difference between sound and lucky. The process judge is built out of exactly that difference.
Why regulators will find this familiar
Grading the steps is not a novel idea in supervision; it is the whole method. An examiner reviewing a credit decision does not check whether the loan was repaid — that verdict arrives years too late and proves little. The exam checks whether the analysis was sound at the time: elements addressed, evidence cited, policy applied. Fair-lending review, SR 11-7’s effective challenge, quality assurance on adjudications — all are process supervision applied to human reasoning, adopted for the same reason the paper documents: outcomes are a noisy, delayed, and gameable signal of decision quality.
What the paper contributes is evidence that the same principle holds inside the machinery: if you want an automated check on machine reasoning, checking the reasoning beats checking the answer. This is the research-side echo of an architectural choice we have argued for directly — reasoning decomposed into steps that can each be verified, with gates between them — in Measuring Reasoning Quality and Artificial Intelligence Is Not Artificial Reasoning. A step can only be graded if it exists as an inspectable unit. A 4,000-token narrative is one step wearing a costume.
Limits, honestly stated
Three cautions before anyone over-generalizes. First, the domain is competition mathematics — a setting where step correctness is unusually well-defined; the authors themselves frame broader applicability as expectation, not demonstration, and a Reg E element or a sanctions judgment is not a algebra step. Second, process labels are expensive — 800,000 human judgments — which is why the paper’s active-learning economies matter and why, in production settings, step-checking must be designed into the architecture rather than annotated after the fact. Third, a process-supervised judge is still a model: it improves the odds of catching flawed steps; it does not certify them. Our own position stays where it has been — these mechanisms reduce and interrupt failure modes; they do not eliminate them.
The takeaway for a compliance reader
When a vendor says their AI is “highly accurate,” the question this paper arms you with is: accurate at the step level or the outcome level — and can you show me the steps? Outcome accuracy on a validation set is compatible with reasoning you would never accept from a human analyst. If the steps cannot be produced, they cannot be graded — by their systems or by yours. The research now says plainly what supervision has always practiced: the step is where quality lives.