How to Read a Reasoning Trace
A machine reasoning record lands on your desk — the account of how an automated system reached a decision now under review. Where do you look first? Not the first page. A trace is written to be read forward, and read forward it recruits you into its own narrative: by the time you reach the conclusion, you have absorbed the premises that make it sound inevitable. What follows is a reading procedure in six passes, starting at the end, and four features that should make a reader uneasy.
Why not start at the beginning
Sequences are persuasive. A competent trace presents its steps in order, each plausibly following the last, and a reader moving through it experiences that plausibility as support. It is not. Coherence is a property of the telling; support is a property of the relation between a conclusion and the evidence under it, and the two can come apart with no seam showing.
Start at the end and the question inverts. Here is a determination, and here is what the record says it rests on. Does this document hold that up? Every element must earn its place rather than merely appear in the right order. You stop following an argument and start auditing one.
The procedure, in six passes
Each pass is a separate read with one question; doing them at once is how readers end up back inside the narrative.
- Read the conclusion and its stated basis — then work backward. Write down the determination and the elements the record claims support it, before reading anything else. Then, for each, find the step that established it and ask whether that step establishes it. Elements that trace to nothing are the finding.
- Check that the inputs were sealed. What evidence was admitted, when, and from where? The record should let you enumerate the evidence set as it stood when reasoning began, and flag whatever arrived after. An open evidence set means the decision’s scope cannot be stated, so it cannot be reproduced or compared to its neighbors.
- Walk the steps and ask what each one decided. Two questions per step: what did this determine, and what would have happened downstream had it come out the other way? A step whose alternative outcome changes nothing is decoration, and a trace mostly of decoration is a narrative with numbered paragraphs.
- Find the gates, and read their verdicts — including the passes. A gate is a checkpoint that can stop a decision; check that each recorded an outcome rather than merely existing. Then widen past this file: a gate with no recorded failures anywhere in the population is not a gate, but a label on a place where nothing happens.
- Look for the abstentions. A well-governed system has decisions it declined to make — insufficient evidence, a case outside the described operating conditions. Find them here, or in the population if not here. A system that never says it does not know is not being careful; it is being silent, and silence and confidence read identically.
- Check the version stamps against the decision date. Model version, instruction and rule-corpus version, configuration — do they match what was in production the day the decision issued? A record stamped with versions postdating its own decision has been regenerated, and it accounts for today’s system rather than the one under review.
Six passes sounds heavy; in practice the first two dispose of most files. A record that survives all six is not necessarily a good decision, but it is one you can argue with.
Four things that should make you uneasy
None is disqualifying alone. Each signals that the document may describe a decision rather than constitute one.
Prose that explains rather than steps that decide. If the substance lives in exposition while the numbered structure carries only labels, the structure was added to the explanation rather than extracted from it. Where no step has a live counterfactual, the prose is doing the work — and prose is not inspectable the way a step is.
Uniform confidence across heterogeneous evidence. An audited figure and an inferred one do not deserve the same footing. A record that states every determination in the same register lost that distinction before the page.
Citations that resolve to documents rather than passages. A reference to a forty-page policy is not a citation; it is a direction to go looking. The question a citation must answer is which sentence.
A reviewer attestation with no recorded questions. Oversight leaving no trace but a signature is indistinguishable from oversight that did not occur. What makes an attestation meaningful is what the reviewer asked, changed, or refused — the friction, not the approval.
What the procedure cannot do
Reading well tells you whether a decision is inspectable and challengeable, not whether it was right — a record can survive all six passes and still rest on a step where the judgment was wrong in a way no structure would reveal. Domain expertise is not replaced by any of this. What the procedure reduces and interrupts is narrower: being carried by a narrative, mistaking coherence for support, taking a reconstruction for the original. Model risk management guidance (SR 11-7) treats documentation as evidence of control, not paperwork beside it. This is that principle, one file at a time.
The computation, or a story about it
Everything above reduces to one distinction. In some systems the trace is the computation: the steps determined the outcome, the gates could have stopped it, and no conclusion exists apart from the path that produced it. In others the trace is a story told about a computation that happened elsewhere — composed afterward, plausible, free to be wrong about its own causes with no visible sign. The two look alike on the page, and the six passes are how you tell them apart: each asks whether something here could have come out differently, and whether it would have mattered.