Your CEO says: “Build an AI that reviews quality events and tells us whether we need a CAPA.”

Your junior engineer writes a mega-prompt. It asks Claude to extract facts from the deviation report, search 500 SOPs, interpret GMP procedures, assess patient risk, determine CAPA necessity, suggest corrective actions, write an investigation report, produce an executive summary, generate an audit trail, and list citations — all in one call.

It fails. Not because Claude is incapable, but because you asked a single model call to perform information extraction, retrieval, legal interpretation, scientific reasoning, risk assessment, planning, and writing simultaneously. That is not a prompt problem. It is an architecture problem.

The fix is decomposition: systematically breaking a complex, ambiguous task into smaller, deterministic, independently solvable sub-tasks that can each be verified before the results are composed back together.

This is the single most tested concept in AI architecture. It is the difference between a fragile demo and a production system.

The Core Principle

Anthropic’s own guidance states it plainly:

Start simple, add complexity only when it demonstrably improves outcomes, and maintain simplicity in your agent’s design.

Decomposition is how you do that. You trade latency and cost for accuracy and reliability by making each LLM call an easier task. Every stage becomes dramatically simpler, more testable, and more auditable.

The 5 Patterns You Must Know

These five patterns come from Anthropic’s Building Effective Agents guide. You need to be able to distinguish them on sight — on the exam and in a design review.

1. Prompt Chaining — Sequential Decomposition

Break a task into a fixed sequence of steps. Each LLM call processes the output of the previous one. You insert programmatic gates (code checks, not more LLM instructions) between steps.

Marketing Copy → Translate → Compliance Check → Format

When to use: The task has a predictable, repeatable workflow. You know the steps before you start.

The gate is the critical detail. The exam loves asking “Where do you add a check?” The answer is almost always between chain steps, with deterministic code — not with another LLM instruction. If step 2 produces JSON, validate the schema with code before passing it to step 3. Do not ask step 3 to “check if the input is valid.”

Trade-off: Added latency. Each link is a full round-trip to the model.

Exam pattern: Outline → criteria check → write document. Research → draft → citation check. Extract → classify → route.

2. Routing — Classification-Based Decomposition

Classify the input first, then direct it to a specialized downstream handler. Each handler has its own prompt, tools, and possibly a different model.

Customer Email → Classifier → ┌→ Billing Agent
                               ├→ Technical Support Agent
                               └→ General Inquiry Agent

When to use: Inputs are heterogeneous. Distinct categories need fundamentally different handling.

The cost optimization angle: Route easy questions to a fast, cheap model (Haiku) and hard questions to a capable model (Sonnet or Opus). This is a tested exam scenario.

The tool overload fix: If you have 50+ tools and the model keeps selecting the wrong one, the answer is NOT better tool descriptions. It is routing combined with dynamic tool scoping — each specialized handler only sees the tools relevant to its domain.

3. Parallelization — Data and Capability Decomposition

Run multiple LLM calls simultaneously and aggregate the results programmatically. Two flavors:

Sectioning — break a task into independent subtasks:

Document → ┌→ Security Scanner (read-only tools)
            ├→ Style Checker (read-only tools)
            └→ Correctness Analyzer (read-only tools)
            → Aggregate Results

Voting — run the same task multiple times for higher confidence:

Code Snippet → ┌→ Vulnerability Review #1
                ├→ Vulnerability Review #2
                └→ Vulnerability Review #3
                → Majority / Threshold Consensus

When to use: Subtasks are independent and can run in parallel, or you need multiple perspectives for accuracy.

The hidden benefit: Context isolation. Each subagent gets its own context window. This avoids the “lost in the middle” problem where a single model processing a long document reliably attends to the beginning and end but misses findings buried in the center.

4. Orchestrator-Workers — Dynamic Decomposition

This is the star of the professional exam. A central LLM (the orchestrator) dynamically breaks down a task based on the specific input, delegates subtasks to worker LLMs, and synthesizes their results.

User Request → Orchestrator (Plan Mode)
                    ├→ Worker A: Search auth code
                    ├→ Worker B: Search DB layer
                    └→ Worker C: Search UI components
                    → Synthesize → Final Output

Key difference from parallelization: The subtasks are NOT predefined. The orchestrator decides what to delegate at runtime based on what the input requires. This makes it flexible where parallelization is rigid.

When to use: You cannot predict the subtasks in advance. Classic examples: coding products that make complex changes across unknown files, research tasks that gather information from multiple sources, or debugging sessions where the root cause could be anywhere.

The exam trap: If a question says “the number of files to change depends on the user request,” do NOT pick parallelization with a fixed list. Pick orchestrator-workers.

How it maps to production: The orchestrator is the main agent session with plan mode. Workers are subagents spawned with their own context windows and toolsets. The filesystem or shared state is the coordination substrate.

5. Evaluator-Optimizer — Iterative Decomposition

One LLM generates. Another evaluates against clear criteria. The loop repeats until the output passes.

Generator → Output → Evaluator → Pass? → Final Output
                ↑                   │
                └── Feedback ───────┘

When to use: You have well-defined evaluation criteria and iterative refinement measurably improves quality. Classic examples: literary translation (the evaluator checks fluency and accuracy), code generation where tests serve as the evaluator, and complex research where the evaluator decides whether further search is warranted.

The stopping condition matters. Do not use arbitrary iteration caps. Inspect the model’s stop_reason (tool_use vs end_turn) to determine loop termination. Parsing natural language signals from the assistant’s text is fragile and unreliable.

The Decision Framework

For every scenario, run this five-question checklist:

Question If yes →
Are the steps known in advance and predictable? Prompt Chaining
Are the inputs heterogeneous with distinct categories? Routing
Are the subtasks independent and parallelizable? Parallelization
Can you not predict the subtasks until you see the input? Orchestrator-Workers
Do you have clear evaluation criteria and iterative improvement? Evaluator-Optimizer

Most production systems combine multiple patterns. A common architecture chains routing into orchestrator-workers, with an evaluator-optimizer loop on the code generation step.

The Contract Between Sub-Tasks

This is where most decomposed systems break down. The exam prioritizes deterministic solutions over probabilistic ones:

Structured output with machine-usable IDs, not prose. If tool A needs to call tool B, tool A must return document_id, not a URL or title. This pattern is tested repeatedly.

Each subtask defines: objectives, output schema, available tools, scope boundaries, and success criteria.

The orchestrator receives summaries, not full transcripts. Verbose worker output consumes tokens disproportionately to its relevance. A PostToolUse hook should trim results before they enter conversation history.

Provenance tracking: Capture stated_value + calculated_value + source_location instead of collapsing information early. This supports debugging, evaluation, and — in regulated industries — audit trails.

Anti-Patterns That Will Sink You

The God Agent

One agent. Fifty tools. Full codebase access. A 10,000-token mega-prompt that tries to do everything.

Why it fails: Attention dilution. Wrong tool selection. Unpredictable behavior. Impossible to test or debug.

The fix: Decompose and dynamically scope tools. Each sub-agent only sees the tools relevant to its domain.

Over-Decomposition

Twelve subagents for a task that a single augmented LLM call with good retrieval could handle.

Why it fails: Coordination overhead exceeds the value. Context fragmentation. Latency explosion. More time spent on integration than on the actual work.

The fix: Start simple. Anthropic’s principle: add complexity only when it demonstrably improves outcomes.

Prompt-Only Fix for a Structural Problem

Adding “be careful with math” to the prompt instead of adding a calculated_total field and a code gate.

Why it fails: LLMs are probabilistic. Asking them to “be careful” does not make arithmetic deterministic.

The fix: Use structured fields and deterministic code gates between chain steps.

Synchronous Loop at Scale

Processing 50,000 documents through a synchronous evaluator-optimizer loop because you need iterative refinement.

Why it fails: Cost and latency explode. You will time out or bankrupt the project.

The fix: Refine on a sample interactively, then deploy the refined prompt via Batch API for the full dataset.

Overly Narrow Decomposition

The orchestrator decomposes “AI impact on creative industries” into digital art, graphic design, and photography. The subagents execute perfectly. The final report only covers visual art.

Why it fails: The coordinator’s decomposition was too narrow. The subagents did their job — the architecture failed them.

The fix: The coordinator owns coverage. If the decomposition is too narrow, the failure is upstream, not in the workers.

Context Management: Why You Decompose Beyond Logic

One reason to decompose is not just logical separation — it is context window economics.

Tool outputs accumulate fast. If lookup_order returns 40 fields and you only need 5, trim the result before it enters conversation history.

Scratchpad files counteract degradation. In long sessions, agents should maintain scratchpad files recording key findings and reference them for subsequent questions.

Subagent delegation preserves main context. Spawn exploration subagents for verbose discovery phases so the main agent retains high-level coordination context.

The “lost in the middle” effect is real. Claude reliably processes the start and end of long inputs but may miss findings buried in the middle. Decomposition directly addresses this by breaking long documents into focused, parallel-analyzed sections.

A Life Sciences Example: Deviation Investigation

Rather than a single “Deviation Investigator” agent, decompose into specialized reasoning stages:

Stage Responsibility Inputs Outputs
1 Fact extraction Deviation report Structured facts
2 Timeline reconstruction Facts + logs Event sequence
3 Document retrieval Facts Relevant SOPs, batch records
4 Compliance analysis Facts + SOPs Procedure deviations
5 Scientific assessment Facts + manufacturing context Technical hypotheses
6 Root cause analysis Prior outputs Ranked root causes
7 Risk assessment Root causes Patient/product risk
8 CAPA recommendation Risk assessment Proposed CAPA
9 Evidence verification All prior outputs Traceability report
10 Human review package Verified outputs Audit-ready dossier

This architecture aligns naturally with regulatory expectations. Every conclusion is traceable to evidence. Every stage can be independently validated. Human review occurs at defined control points. The system produces an audit trail by construction, not as an afterthought.

Decomposition Is Not Always the Answer

Simple, single-shot tasks do not need decomposition. If a single well-crafted prompt handles the task reliably, adding orchestration only adds cost, latency, and failure surface.

Well-structured modular systems can be over-decomposed. If breaking a service apart requires five network calls to complete a single transaction, you have created a distributed monolith with none of the benefits and all of the complexity.

Start simple. Anthropic’s own principle. Add complexity only when it demonstrably improves outcomes.

The Bottom Line

Decomposition is the hallmark of moving from a prompt engineer who writes clever single prompts to an architect who designs resilient, distributed AI systems.

When you see a complex task, your first instinct should be: How do I split this into smaller, verifiable, and repeatable steps?

Pick the pattern that matches the problem’s structure — fixed steps get chaining, unknown steps get orchestrator-workers, heterogeneous inputs get routing, accuracy-critical tasks get voting, iterative improvement gets evaluator-optimizer. Define structured contracts between steps. Add deterministic gates where stakes are high. Preserve context through isolation, not accumulation.

If you can draw the flow, justify why you chose each pattern, and identify where validation belongs, you have internalized the core skill that production AI architecture demands.