Why Multi-Agent AI Systems Fail: A Field Guide to the MAST Taxonomy

The paper

Wire several model-driven agents together — a planner, workers, maybe a reviewer — and let them collaborate on a task. The architecture is everywhere in current pilots, and the marketing implies the ensemble is more reliable than any single model. This paper asked the empirical question: when these systems fail, how? The authors had expert annotators read 150+ execution traces from 7 popular multi-agent frameworks, labeling each failure they found, then organized the labels into a taxonomy: MAST, 14 failure modes in 3 categories.

The categories are worth internalizing, because each names a different layer of the system. Specification and system-design failures: the task or the agents’ roles were defined badly enough that failure was arranged before any agent acted — ambiguous objectives, roles that overlap or leave gaps. Inter-agent misalignment: the agents fail each other — miscommunication, inputs one agent supplies and another ignores, conversations that derail away from the task. Task verification and termination failures: the checking layer fails — work is verified weakly or not at all, and processes declare completion prematurely, before the task is actually done. Across the annotated traces, failures were widespread even on simple tasks — and weak verification emerged as a leading contributor. The reviewer agent so many frameworks include was, in practice, often a rubber stamp.

An examiner’s checklist, arriving just in time

Agentic pilots are moving into banks — operations, service, increasingly workflows that touch regulated determinations — and reviewers will need something sharper than a demo to assess them. The taxonomy’s three categories map directly onto three requests for evidence. For specification failures: show the written contract for each agent — its objective, its inputs, its boundaries — and show that the decomposition was reviewed the way any procedure is reviewed, because an ambiguity in an agent’s role is a defect in a control document, not a model quirk. For inter-agent misalignment: show what one agent is required to pass to the next, and what in the architecture — rather than in good intentions — makes a handoff that drops or ignores information detectable, since an unmonitored conversation among agents is exactly where derailment lives. For verification and termination failures: show the check that stands between an agent’s output and its consumption downstream, what criteria it applies, and what happens on failure — and show why “done” is a verdict something renders rather than a state an agent announces.

The category a governed architecture answers

The third category is the one an architecture can most directly close, and it is the one our own design is organized around. In an IRG, verification is not an agent politely reviewing a colleague’s work inside the same conversation; it is a gate — a distinct step with stated criteria whose failure blocks the output from proceeding. Between components, nothing is consumed that has not passed its gate; where drafts conflict, an arbiter step resolves them under criteria of its own rather than by whichever agent spoke last; and termination is a gate verdict, not an announcement — a run ends when convergence criteria are met or an abstention path fires, never because a participant declared victory. This does not abolish the first two categories: specification quality is still human work, and the paper is useful precisely because it shows how often that work is skipped. But it relocates verification from the culture of the ensemble into its structure — the difference we have described as active rather than passive governance — which is what makes the weak-verification failure modes reduce and interrupt rather than accumulate silently.

Limits, honestly stated

The taxonomy’s scope deserves stating plainly. It was built from 150+ traces across 7 frameworks — a substantial annotation effort and still a sample, drawn from the open frameworks of a fast-moving moment; production systems in regulated institutions may fail in ways the sample underrepresents, and framework versions evolve. Failure annotation is expert judgment, and boundaries between 14 modes will blur on real traces. And a taxonomy is a diagnostic instrument, not a cure: naming a failure mode does not prevent it, and a gated architecture addresses the verification category directly while only narrowing the surface of the other two — a well-gated graph built on a bad specification will faithfully verify its way to the wrong objective. As ever: these mechanisms reduce and interrupt failure modes; they do not eliminate them.

The takeaway for a compliance reader

When an agentic pilot reaches your desk, MAST gives you the structure of the review: three categories, three demands for evidence — specification contracts, handoff requirements, and verification with the power to block. The paper’s empirical warning is the part to keep: failures were widespread on simple tasks, and the weakest link was checking. Ask where verification lives in the architecture, what it can actually stop, and who verified the verifier. If the answer to the first question is “one of the agents,” the taxonomy has a category waiting for it.