AI Agents vs Workflows? Count Your Branches First

Most requests for an agent are workflows with an LLM inside, and you can settle the AI agents vs workflows question on a whiteboard before anyone writes code. The pitch sounds open-ended, but when you enumerate the actual control-flow branches, the tree closes fast: route, extract, validate, escalate, done. What remains is a pipeline with a model at one or two steps.

So the decision is not a taste call. It reduces to arithmetic on runtime decision points, the moments where a model chooses what happens next instead of code that already knows. Every branch you delegate to a model adds token spend, adds seconds of tail latency, and multiplies the ways a run can fail. Every branch you keep in code costs nothing at runtime and fails in predictable, testable ways.

The test below takes 30 to 60 minutes and runs before architecture is chosen. You enumerate every branch, price each runtime decision point, and promote to an agent only when the branch set cannot be known in advance. You finish with a decision ledger for your own feature, five worked examples that land on both sides, and a demotion playbook for the agent that looked inevitable in the demo.

AI Agents vs Workflows, Defined by Who Owns the Branches

Anthropic's engineering guide draws the line cleanly: a workflow orchestrates LLMs and tools through predefined code paths, while an agent dynamically directs its own process and tool use. The phrasing matters less than what it implies about ownership. In a workflow, every branch point is resolved by code someone wrote in advance. In an agent, branch decisions are handed to the model at runtime. Who owns the if drives every downstream difference in cost, latency, and reliability that this article prices.

The term agentic workflows muddies this. It gets applied to everything from one LLM call inside a cron job to a fully autonomous loop, while business publications keep refining the umbrella label, as HBR's primer on agentic AI shows. For engineering purposes, collapse the vocabulary. If code owns the branches, it is a workflow, even if five LLM calls live inside it. If the model owns any branch that materially changes the path, agent behavior has entered the system and the arithmetic in Step 2 applies.

The split has also reached the tooling. LangGraph's orchestration docs treat workflows and agents as distinct constructs rather than points on one continuum, a signal that the difference is structural, not stylistic.

Default position: workflow. Anthropic's own guidance is to find the simplest solution possible and add complexity only when it demonstrably helps, and most features pass through that gate unchanged.

The Decision Test Before You Write Code

Run this before you pick a framework, before the design review, ideally before anyone says "agent" with conviction in sprint planning. You need a whiteboard and one person who knows the task cold. If you have been wondering how to decide between an agent and a workflow for a specific feature, this sequence is the entire AI agents vs workflows decision procedure:

1. Enumerate every control-flow branch the feature must take, including error paths and escalations.
2. Price each runtime decision point for cost, latency, and reliability at realistic volumes.
3. Apply the promotion criteria, and promote only if the branch set cannot be known in advance.

The test answers exactly one question: is any branch in this feature genuinely unknowable in advance? If yes, you have an agent-shaped hole, and Step 3 covers how to fill it safely. If no, you have a workflow, and the remaining work is deciding which fixed branches a model should route.

Step 1. Enumerate Every Control-Flow Branch

Write down every decision the feature must make, then sort each into three buckets:

Branch type	Example	Rightful owner
Static	If invoice total exceeds the threshold, route to approval	Code, always
Model-routable	Which of these 12 intents matches the ticket?	Code, with an LLM classifier choosing among a fixed set
Unbounded	Given what this search returned, what should I read next?	Nobody can list the set in advance. This is where agents live

Most features enumerate into a dozen static branches, two or three routable ones, and zero unbounded ones. That profile is a workflow with a router, a solved and testable shape.

Rule of thumb: if you can draw the complete if/else tree on the whiteboard, you need a router plus a prompt, not an agent.

A router over a fixed branch set is a classification problem. That means it is measurable against a labeled set, cheap to run, and swappable for a fine-tune or a rules pass later. An agent over the same set buys nondeterminism and pays for it in Step 2.

One honest wrinkle: a tree that enumerates but closes at 200 branches with hostile overlap has an enumeration cost of its own. Hold that thought, because enumeration fragility is an explicit input to Step 3.

Step 2. Price Each Runtime Decision Point

AI agent cost is decision-point arithmetic times volume. The cost per LLM decision point is:

cost per decision = (input_tokens × input_price + output_tokens × output_price) / 1,000,000

Take a modest routing decision: 1,500 tokens in (system prompt, branch schema, two few-shot examples) and 50 tokens out. At an illustrative $3 per million input and $15 per million output, that is roughly $0.005 per decision. Current numbers live on OpenAI's pricing page and Anthropic's pricing page, and both move often enough that you should recompute rather than trust any article's figures, including this one.

Scale makes the rounding errors loud. 100,000 requests a day with one delegated decision each is about 3 million decisions a month, on the order of $15,000 at the illustrative rate, spent reproducing an if statement that was free in code. Small-model tiers cut that by an order of magnitude, which is exactly the point of pricing per decision point: the answer is a knob, and you should know you are turning it.

Latency stacks the same way. Mean response time grows at least linearly with the number of sequential LLM calls, and the p95 climbs faster than the mean because one slow call sets the pace for the whole chain. Five chained calls at two seconds each is a ten-second average wait before retries even exist.

Retries and loop guardrails multiply the bill. A step that fails 20 percent of the time and retries once adds 20 percent to expected spend, and a loop capped at five iterations means one confused run can consume five times its budget. None of this shows up in the demo.

Reliability is the part teams underweight hardest, because success compounds multiplicatively across decision points:

Sequential decision points	Per-step success	End-to-end
5	95%	~77%
5	90%	~59%
10	95%	~60%
10	99%	~90%

Five decision points at 95 percent each land near 77 percent end to end, before you count tool failures, parsing errors, or truncation. Surveys of LLM agents keep cataloguing planning and tool-use failure modes that surface in multi-step runs; this multi-step error compounding, not single-call accuracy, is what separates agent reliability from workflow reliability.

The asymmetry is the entire cost model. Delegated branches multiply cost, latency, and failure modes; branches in code multiply nothing.

Step 3. Apply the Promotion Criteria

Everything above collapses the question of when to use AI agents into three conditions. Promote only when one holds:

1. The branch set cannot be enumerated in advance at acceptable cost. Open-ended search, multi-step research, novel tool chains. You cannot draw the tree because the next branch depends on what the model just found.
2. Enumeration is possible but more fragile than the agent premium. A 200-branch router maintained against drifting inputs can burn more engineering time and cause more misroutes than a bounded agent spends in tokens.
3. Failures are cheap to catch. Verifiable outputs, sandboxed tools, human review of the tail. An agent whose mistakes are expensive needs code rails regardless, which usually means it belongs inside a workflow anyway.

Vendor guidance converges on restraint from the same direction. OpenAI's practical guide walks the ladder from single calls through workflow patterns before agents, and AWS Bedrock's agent guidance frames agents for multi-step tasks that plan and call tools dynamically, not as a default wrapper for any LLM feature.

Then there is the pattern that deserves to be your default ambition: a workflow shell with one agent-shaped hole. Known branches, intake, validation, budget caps, escalation, formatting, stay in code, and exactly one unbounded segment gets delegated. The shell enforces the loop cap, the spend ceiling, and the fallback path; the agent inside does the only thing code cannot, which is choose the next question based on what it just learned. This hybrid architecture, a workflow shell around one agent-shaped hole, is how serious research features ship, and it is the LLM agent architecture that survives contact with production budgets.

Five Worked Examples, From Triage to Research

Invoice and document extraction. Fixed schema, fixed validation rules, an exception queue for humans. Every branch is static. The LLM extracts fields; code owns everything else. Verdict: pure workflow.

Support triage. The workflow or agent question for AI support triage straddles a single line. Intent classification and routing across a known catalog is a bounded router problem. Messy resolutions, the ticket that requires checking three systems and negotiating a policy edge case, are unbounded. Verdict: workflow with an agent-shaped tail. Route with code plus a classifier, delegate only the residue, and cap its spend.

Code review triage. Parse the diff, run linters and tests, classify severity against a known rubric, notify the right channel. The branch set is small and stable. Verdict: mostly workflow. The judgment call on borderline severity is a classifier step, not a license to explore.

Deep research. The next query depends entirely on what the previous results contained. Nobody can draw the branch tree in advance, including the team that built the feature. Verdict: genuinely agent-shaped, and still best delivered inside a workflow shell that owns budgets and citations.

Autonomous ops remediation. The self-healing infrastructure pitch. Most incidents match known runbooks, the branches are enumerable, and the cost of a wrong remediation is high. The demo is always the novel incident; production is always the common one. Verdict: fails the test. Ship a workflow plus router for known runbooks, keep an agent-shaped hole for the true residue, and gate anything with teeth behind a human.

When and How to Demote an Agent Back to a Workflow

Some agents get built anyway. A failing one announces itself in telemetry long before anyone says so in a meeting:

• Cost per resolution climbs while volume mix stays flat. Same tickets, more tokens per close.
• Median loop count creeps up. Tool-call iterations per task drifting from three to six to nine means the model is thrashing, not reasoning.
• Success rate sits below a fixed baseline. If a single-call pipeline or a rules pass beats the agent on your eval set, the agent is decoration.
• Human override rate rises. Every approval is a demotion vote cast one at a time.

Demotion is cheap, provided you did one thing at build time: log every model decision, the route, the tool, the arguments, the outcome. Those logs are a census of the real branch distribution, the tree your traffic actually walked. To demote an agent to a workflow:

1. Mine the decision logs and rank branches by observed frequency.
2. Freeze the dominant branches into code. The top handful typically covers the large majority of traffic.
3. Keep the agent as the fallback path for the residue, behind its budget cap.
4. Shadow-run the frozen workflow against live traffic before switching, then compare cost per resolution and success rate.

Writing control flow down and owning it outright is a running theme of the 12-factor agents principles, and it is what makes this reversible. If demotion looks expensive, that is not an argument for the agent; it is a sign the agent was never instrumented.

The Pre-Commit Checklist and Decision Ledger

Take this to the design review:

• Every control-flow branch enumerated and classified as static, model-routable, or unbounded
• Runtime decision points counted, with cost per decision computed at current list prices
• End-to-end reliability multiplied out at realistic per-step accuracy
• Failure cost assessed per branch, meaning the price of one wrong turn
• Loop caps, spend ceilings, and fallback paths specified before launch
• Telemetry defined up front: cost per resolution, loop counts, override rate, success versus a fixed baseline
• A named owner for the demotion decision and a date to revisit it

And the ledger, one row set per feature under discussion:

Ledger field	Your entry
Static branches	count
Model-routable branches	count
Unbounded branches	count
Runtime decision points	count
Cost per decision point	dollars
Expected end-to-end success	percent
Agent-shaped hole	yes or no
Verdict	workflow, hybrid, or agent

Reopen the ledger when the ground shifts: the task distribution changes materially, a model upgrade moves the cost or accuracy lines, or a new tool collapses a formerly unbounded branch into a routable one. Promotion is not permanent, and neither is demotion.

The branch test compresses the whole AI agents vs workflows debate into one question you can answer with a marker: can every branch be known in advance? When yes, ship the workflow and let the model do the narrow thing it is good at inside it. When no, cut exactly one agent-shaped hole, cap its budget, and log every decision it makes, so the day it loses to a pipeline, you have the receipts.