The Examiner Is the User

The user who arrives later

Product design has a reflex: identify the user, study their workflow, minimize their friction. Applied to AI in the enterprise, the reflex produces tools optimized for the person at the keyboard — faster drafts, cleaner summaries, fewer clicks between question and answer. Every demo is staged from this user’s chair. Every metric — latency, satisfaction, adoption — is measured from it.

But consequential workflows have a structural feature that consumer software does not: the output outlives the interaction, and its most important reader hasn’t arrived yet. The credit memo will be read by a fair-lending examiner pulling a sample two years from now. The dispute denial will be read by a consumer-compliance examiner, or a plaintiff’s attorney with a document request. The validation report will be read by a model-risk supervisor; the SAR narrative by a federal investigator; the coverage determination by a regulator reconstructing how a class of claims was handled. This second user has properties the first never has. They read one file at a time, completely. They compare it against the record and against how similar cases were treated. They are professionally obligated to be adversarial. And they hold the only score that ultimately matters: whether the institution can keep running the system at all.

Two users, two designs

Ask what each user needs and you get nearly opposite specifications. The operator wants an answer — concise, confident, fast; justification on demand at most, since reading it is friction. The examiner wants an account: what was considered, in what order; what evidence supported each conclusion and where it came from; what the system was uncertain about and what it did about the uncertainty; whether this case was treated like the cases next to it; whether anything was checked, challenged, revised. The operator experiences hedging as noise. The examiner experiences unhedged confidence as a finding. The operator’s ideal output is one paragraph. The examiner’s is a file.

Systems designed purely for the operator fail the examiner in a characteristic way: fluently. The output reads as considered judgment, but when the second user asks the second-user questions — show me the basis; show me the alternatives you ruled out; show me this decision is consistent with the last hundred — there is nothing behind the prose. Not because the model reasoned badly, but because nothing preserved the reasoning, checked it, or bound the confidence of the prose to it. The institution is left defending a conclusion with a reconstruction — which any examiner recognizes as the difference between evidence and testimony.

This, we’d argue, is the real explanation for AI’s perimeter problem. Enterprises did not keep AI out of credit, coverage, and triage because the models lacked capability. They kept it out because every deployment was designed for user one, and the institution’s risk, legal, and compliance functions — who answer to user two — correctly refused to sign. The workflows AI hasn’t entered are precisely the ones where the second user’s verdict is decisive.

Designing for the second user

Take “the examiner is the user” seriously as a design brief and the architecture follows. The reasoning must persist — not as a chain-of-thought transcript, but as a structured record of steps, evidence bindings, checks passed, and revisions made, sealed at decision time rather than reconstructed later. Verification and challenge must be part of the workflow, because the examiner’s first question is never “what did you conclude” but “what would have caught this if it were wrong.” Uncertainty must be handled as a first-class outcome: an abstention with a documented reason is a good file; a confident guess is a liability, even when it happens to be right. Consistency must be inspectable across cases, since like-cases-treated-alike is the second user’s core test. And the numbers must be reproducible — anything an examiner can re-run should re-run to the byte.

Here is the part the product instinct gets wrong: none of this degrades the first user’s experience. It rescues it. The operator in a regulated workflow was never actually free to accept a bare answer — they were the one assembling the defensible file by hand afterward, which is where their hours actually went. A system that produces the examiner-grade artifact as a byproduct of reasoning gives the operator the thing they truly wanted: an answer they are allowed to use. Speed-to-answer was always the wrong metric in these domains. Speed-to-defensible-answer is the real one, and designing for the second user is the only way to win it.