AI Agent Orchestration, Thin Router or Fat God Agent

Somewhere between the first demo and the third production incident, most agent teams converge on the same component: one central loop that knows the schemas, the retry rules, the naming conventions, and the escalation thresholds. Meta's engineering team recently published the countermove, a 1,300-line orchestrator that routes more than 20 workflows to 14 specialized agents and performs zero domain work itself.

The lesson worth taking runs deeper than "small routers are virtuous." AI agent orchestration is a knowledge-placement decision: procedural knowledge belongs in specialized executors, value judgments belong in policy files the router consults, and route selection becomes deterministic code. Get the placement right and the router shrinks as a side effect. This piece quantifies what a fat loop costs at runtime, rebuilds Meta's router as a copyable spec, argues the honest cases for staying thick, and ends with a staged migration behind a feature flag.

What AI Agent Orchestration Actually Controls

"Orchestrator" is one word doing six jobs. Before deciding how thin yours should be, enumerate which jobs currently live in the model loop, because each has a natural home elsewhere in your AI agent architecture.

Job	What it decides	Natural home
Route selection	Which executor runs next	Deterministic code (classifier, schema match)
Sequencing	Order and parallelism of steps	Workflow template, a state machine
Context assembly	What each executor sees at dispatch	Typed contract, filled per step
Retries and gates	Timeouts, retries, reviewer checks	Code
State custody	Intermediate artifacts between steps	Durable store keyed by run
Policy enforcement	Thresholds, budgets, approvals	Human-owned policy files

OpenAI's practical guide frames an agent as a model dynamically directing its own tool calls in a loop, and Google's agents whitepaper centers an orchestration layer that maintains memory, state, and reasoning for that loop. The god agent anti-pattern is what you get when all six jobs run inside that single loop's context: the model routes, sequences, assembles, retries, remembers, and judges in one window.

Thinness is therefore measurable, not vibes. Count the six jobs living in model-reasoned paths; that count is your orchestrator's effective thickness.

What a Thick Orchestrator Costs in Production

Thick routers fail in three specific ways, and each compounds as runs get longer.

Context that compounds per hop

In a god-agent loop, every intermediate tool result the model reads stays in its window for all later steps, so spend and interference grow with the number of hops, not the number of steps. Run the arithmetic for a ten-hop task with explicit assumptions:

Architecture	Assumptions	Formula	Ten-hop total
God-agent loop	2,000-token system prompt; 1,200-token average tool result, never evicted	Hop n carries roughly 2,000 + 1,200 x (n - 1)	20,000 + 1,200 x (1+2+...+9), about 74,000 prompt tokens
Thin router dispatch	Each dispatch carries its own executor system prompt of about 2,000 tokens, plus a fixed envelope of around 500 tokens	10 x 2,500	Roughly 25,000 tokens

The god loop's total scales quadratically as hops increase, because every hop re-reads all earlier results. The thin side also changes who pays for knowledge: each specialist pays its own one-time knowledge load, a few thousand tokens per task in Meta's system, once per step instead of once per hop. That is context bloat per hop, and it is the core runtime argument for thinness.

Tokens are an interference surface, not just a bill. Meta's team watched a fully loaded single agent cross-contaminate concepts, applying one subsystem's naming conventions to another, and found that past a threshold, more knowledge degraded output rather than improving it. Meta also cites a Mount Sinai preprint reporting orchestrated multi-agent setups holding 65.3% accuracy on 80-task batches where single agents fell to 16.6%, with up to a 65-fold token reduction, attributed to isolating each task's context. One workload; treat it as directional. Anthropic's context engineering notes argue the same from the design side: attention is finite, so each step should get just enough context and no more.

The blast radius of one mis-route

When routing logic lives in the prompt, one wrong branch is inherited by every downstream executor, which then goes on to answer a different question, correctly. The MAST failure taxonomy from the "Why Do Multi-Agent LLM Systems Fail?" paper organizes observed failures into categories including underspecified designs, misalignment between agents, and weak verification. In plainer terms, the center misjudging and everything downstream inheriting it is a documented, recurring failure class. Deterministic routing fails smaller: the wrong executor receives one step, a gate catches the mismatched output, the step retries, and the rest of the run never hears about it.

The router is still a single point of failure

Thinning does not solve this one. Any component that decides every next step is a single point of failure, thin or fat, and a minimal router can fail less often while failing just as totally. Router minimality and router resilience are separate engineering problems. The mitigation is idempotent dispatch, so a retried handoff cannot double-execute a side effect, plus a deterministic fallback route that degrades to a known-safe path or a human queue when the router itself is down.

Meta's Minimal Router as a Build Spec

Meta's engineering writeup gives enough detail to copy, which is rarer than it should be. Meta calls the result the thin orchestrator pattern, and the agent router design decomposes into three relocations.

What the router keeps. A fixed seven-step protocol run for every request: read the target executor's instruction file, check whether the target already exists, gather required fields, auto-discover what can be inferred, run reliability gates such as active warnings and required reviewers, delegate with full context, then handle results and surface errors. Everything except the delegation step is deterministic code. The router classifies intent and enforces checks. It never reasons about the domain, which is how 1,300 lines cover more than 20 workflows.

What moves down. Fourteen specialists, each a domain expert with an isolated context and a uniform anatomy: knowledge sources, a pre-gathering checklist, a step-by-step workflow, discovery patterns, and error-to-resolution mappings. More than 25 knowledge files load at runtime, a few thousand tokens each, versioned independently, so an update lands in the next session without redeploying the router. This is the orchestrator-worker pattern taken to its endpoint, with the intelligence entirely on the worker side.

What moves sideways. Phase ordering lives in workflow templates, deterministic state machines that sequence the same pool of executors differently per use case. Supporting pipeline migration, a genuinely new workload, meant writing a new template and zero new agent code.

Nothing was deleted. The router knows the shape of every task, which fields, which gates, which order, and the content of none of them. The reported results follow from that placement: configuration prep that took three to four days by hand and about a day under an earlier system now finishes in under an hour, and roughly two-thirds of the system's changes land without significant human edits. A single-agent alternative Meta evaluated hit 70-80% success only when an engineer hand-crafted a prompt per task, and 10-20% without that handholding. Architecture, not prompting.

Knowledge Placement, Down to Executors, Up to Policy

The teardown converts into three placement rules, and together they answer where to put routing logic in multi-agent systems along with everything else.

Rule 1: knowledge that describes how to do one step well moves down. The dialect quirks of your warehouse SQL, the codegen conventions of a specific repo, the retry behavior of a flaky internal API. The test is simple: if it reads like documentation for a specialist, it belongs in that specialist's knowledge file or, better, in its code. Executors that own their procedure can be tested, profiled, and upgraded independently.

Rule 2: value judgments move up into policy. A refund threshold is not procedural knowledge; it is a business decision wearing a technical costume. Kept in prompt text, it resists review, because nobody diffs prompts the way they diff code; it resists versioning, because nobody can say what rule ran last Tuesday; and it resists audit. As a policy file the router reads, it gets pull-request review, unit tests asserting outcomes, and an audit trail with a commit hash:

# refund_policy.yaml, illustrative
auto_approve_under: 50
gold_tier_auto_approve_under: 150
block_auto_if_refunds_90d: 2
always_route_to_human: [chargeback, fraud_flag]

The router's job is to read the policy and route accordingly, which keeps the judgment human-owned and the routing mechanical.

Rule 3: route selection stays, as code. "Which executor handles this" is a classification problem, keyword matching, schema matching, or a small intent classifier, not a reasoning task. If your router must reason its way to the next step, routing is in the wrong layer. Anthropic's building effective agents guidance pushes the same instinct: compose deterministic workflows wherever you can, and reserve model discretion for the steps that genuinely need it.

When a Thick Orchestrator Still Wins

The thin versus thick orchestrator question has honest thick answers, and pretending otherwise just relocates the failure.

Cross-step invariants. Some workflows require each step to consume validated output from the prior step, with validation policy that must not be re-decided at every handoff. A build gate must block a deploy regardless of what the deploy executor believes about its own output. Push that invariant into handoff contracts and every contract re-encodes it, and any one of them can quietly drop it.

Long-horizon plans. Compliance research or incident triage deepens as evidence arrives: finding number 40 changes what to look for in number 41. One global state holder that re-plans beats a library of enumerated templates, because the templates describe a world that has already changed.

Tightly coupled tool chains. A data migration with rollback needs the undo path to know exactly what step three wrote, mid-flight, not a summary of final outputs. A handoff contract big enough to pass all of that is a thick orchestrator with worse ergonomics.

The nuance that matters: thick here means state custody, not intelligence. The defensible thick orchestrator is usually dumb code holding a typed, durable, checkpointable global state machine, with one model call reserved for re-planning. Domain reasoning in the loop is what the god agent anti-pattern actually consists of; one place holding the run's state is sometimes a genuine requirement. Be thick in the state layer and thin in the reasoning layer.

A Router Scope Rubric, Scored on Five Builds

Most arguments about multi-agent orchestration patterns stall because both sides mean different jobs by "orchestrator." AI agent orchestration becomes tractable the moment you score it instead of debating it. Four questions, each with a thin answer and a thick answer:

1. Can each step's inputs and outputs be schema-validated at handoff? Contracts everywhere points thin.
2. Do later steps need mid-flight intermediate state, or only the previous step's outputs? Mid-flight state points thick.
3. Is the plan enumerable as workflow templates, or does it genuinely drift with findings? Enumerable points thin.
4. Can a wrong route be confined by retrying one step? If a mis-route poisons everything downstream, that points thick.

Three or four thin answers, run a thin router over workflow templates. Two or more thick answers, especially with high blast radius, build the thick state holder and keep it dumb and checkpointed. Scored against five representative builds:

Build	Contracts	Mid-flight state	Plan drift	Mis-route blast	Verdict
Support triage	Strong	Rare	Enumerable	Low	Thin; thresholds as policy files
Code-change pipeline	Strong	Yes, gates	Enumerable	High	Thin sequencing, invariants as deterministic gates
Compliance research	Medium	Common	Drifting	Medium	Thick planner, checkpointed
Data migration	Strong	Yes, rollback	Enumerable	High	Thick state holder, dumb steps
Document processing	Strong	Rare	Enumerable	Low	Thinnest; a queue with a classifier

Two rows look wrong at first glance. The code-change pipeline scores thin despite high mis-route blast because its build gates and deploy blocks run as deterministic gates inside the sequencing, not as model judgment; a mis-route is stopped by a contract check, not by router intelligence. The data migration scores thick on the opposite logic: rollback needs mid-flight custody of exactly what step three wrote. Its thickness is state custody, not intelligence.

The pattern across the table: thickness follows state custody and plan drift. In every build scored here, intelligence in the router bought nothing except incident count.

Migrating a God-Agent Loop to a Thin Router

None of this requires a rewrite. Live pipelines can move to a thin router incrementally, via the strangler fig pattern: grow the new system around the old one and let it take over piece by piece.

1. Instrument the loop first. Log every hop: stated intent, chosen executor, payload size, and the branch condition the model applied. After a couple of weeks you own a routing-trace corpus, which becomes the parity baseline for everything after.
2. Shadow-route behind a feature flag. The new thin router classifies live traffic but does not execute; you compare its decisions with the god loop's actual choices. Every divergence is a spec gap enumerated by real data rather than by opinion.
3. Extract executors behind stable contracts. Start with the most self-contained step. Wrap it as an executor with typed input and output plus its own runtime-loaded knowledge file, and let the god loop call it exactly as before. Repeat. Each extraction physically removes knowledge from the god prompt, and the loop gets measurably dumber in the good direction.
4. Promote routing rules to code only at parity. Convert a prompt heuristic into a rule when it reproduces the god loop's observed decisions on the trace corpus within a divergence budget you set before you start, not after.
5. Cut over per workflow, with the safety net. Idempotent dispatch keys on side-effecting steps, a deterministic fallback to the old loop or a human queue if the router fails, and rollback scoped per workflow rather than system-wide. Teams on LangGraph can start from LangGraph's supervisor template, one supervisor node routing worker nodes with state held in the graph, then thin the supervisor from there.

The audit that starts all of this takes ten minutes. Open the router's system prompt and ask, for each rule, which layer should own it. Documentation becomes an executor's knowledge file. Judgments become policy with a commit hash. Whatever survives is routing, and routing is code. If deleting the prompt would destroy domain knowledge, the loop is holding knowledge it should never have had, and that is what Meta's minimal router got right.