Skip to main content
Guides ••11 min read•

Human-in-the-Loop AI Agents That Pause for Hours

Human-in-the-loop AI agents that pause for hours need async design. Compare three resume architectures, from client polling to durable execution engines.

Human-in-the-loop AI agents pause mid-task while a person reviews and approves the proposed action before execution resumes.

A published walkthrough of a Google ADK agent that waits on a human describes a synchronous request holding its HTTP connection open for 83 seconds. The author presents that as a win, but 83 seconds is the floor of the problem, not its edge: a worker, a load balancer, and a client connection all sat parked so one person could click approve. Human-in-the-loop AI agents routinely wait minutes for a Slack reply, hours for a manager, and days for legal review, while nearly every agent quickstart is built for calls that finish in seconds. Once expected wait crosses one to two minutes, the synchronous call is the wrong primitive. The hard engineering moves to the resume path: idempotent side effects, versioned durable state, credential and context expiry, and the design of the handoff itself. That layer sits underneath ADK, LangGraph, and Step Functions alike, and it is the part frameworks document least.

Stay in the loop.

Get the latest posts and exclusive content delivered to your inbox.

Join 13 readers. No spam. Unsubscribe in one click, anytime.

This article covers four things in order: a decision rule for going async, a comparison of the three resume architectures, a catalog of failure modes that only appear once the wait stretches past a coffee break, and a shipping checklist.

Why the Synchronous Call Fails Human-in-the-Loop AI Agents

Everything on the synchronous request path is tuned for interactions measured in seconds. Model SDK clients set request timeouts in that range. Managed gateways and load balancers commonly cut idle connections after tens of seconds to a few minutes. Browser fetches abort, phone networks hand over mid-request, and users close laptops. A human approver does not care about any of this, and the infrastructure does not care about the approver.

Put a human decision inside that path and three costs stack up:

  • Blocked capacity. Every waiting agent occupies a worker, a connection, or an executor slot for the full duration of the wait, doing nothing.
  • Retry storms. When the client or an intermediate proxy times out, the retry often starts a second run of the same intent while the first is still alive. If both eventually execute the approved action, the customer is charged twice, access is granted twice, or the email ships twice. The timeout did not fail gracefully; it manufactured a duplicate.
  • Pool exhaustion. A handful of simultaneous approvals can tie up a large share of a small service's connections or worker slots, a failure mode that only appears under real concurrency.

Once expected human latency exceeds one to two minutes, holding a synchronous connection costs more in timeouts, connection limits, and retry storms than an async design costs in plumbing. Below that threshold, synchronous is fine and simpler. Human approval is almost never below that threshold.

A Decision Rule for Going Async

The question of when to make an AI agent async instead of synchronous has a defensible answer, and it comes from three inputs rather than instinct:

  1. What is the p95 wait? Ask what the slowest approver does on a Friday afternoon, not the median. Legal review and shift boundaries stretch the tail far past the average.
  2. What does blocking cost? A queued background job is cheaper to park than a web handler holding a response, and both are cheaper than a user staring at a spinner.
  3. What happens on client retry? Read-only steps re-run harmlessly. Side effects double. This single question predicts most of your production pain.
Expected p95 waitBlocking costRetry blast radiusVerdict
Under about 60 secondsLow (background worker)Read-only stepsSynchronous is acceptable
1 to 10 minutesAnyAny side effectsGo asynchronous
Over 10 minutes or unboundedAnyAnyAsync is mandatory; prefer durable execution for many-step workflows

Synchronous stays defensible under a narrow set of conditions: bounded sub-minute waits, a single tenant or internal tool, a client you control that can sit and wait, and steps that are already idempotent. If any one of those conditions breaks, so does the design.

Three Resume Architectures Compared

The 202 Accepted webhook pattern for AI agents returns a job receipt immediately and lets a webhook callback resume the run once approval lands.

Once the workload goes async, the real question is who drives the resume. There are exactly three answers, and every framework feature reduces to one of them.

Client Polling

The server accepts the job, stores the intent, returns a job id, and the client polls a status endpoint until the run completes. This is the right answer for demos and internal tools: simple, debuggable with curl, no inbound infrastructure required. The structural weakness is ownership. The client drives the resume, so if the approver takes six hours and the user closed their laptop at minute two, nothing is polling and nothing resumes. Polling also scales connections with waiting users, which quietly recreates the load problem you were escaping.

Webhook plus a 202 Accepted Receipt

The server persists the intent and returns a 202 Accepted status code with a receipt: the request was accepted for processing, and here is a job id for checking on it later. The approval happens wherever the approver lives, and that surface calls your webhook callback when the decision lands. The agent then resumes server-side. The client can still poll or subscribe for UX, but the resume no longer depends on the client surviving the wait, which is why the 202 Accepted webhook pattern is the default production choice. What you inherit is webhook machinery: signature verification, idempotent handling of redelivered events (delivery systems commonly retry), and ordering.

Durable Execution Engines

AWS Step Functions' waitForTaskToken pauses a state machine until an external system returns a task token, which maps one-to-one onto human approval. Temporal and Azure Durable Functions generalize the idea, and Microsoft's durable task human-interaction pattern walks through the durable timers and escalation you would otherwise hand-build. With an engine, execution state, timers, retries, and versioning live in the platform, so resume after hours or days is a platform guarantee rather than your code. The price is a new stateful system to operate and a real learning curve for the team.

Client pollingWebhook plus 202Durable execution engine
Build costLowMediumMedium to high
Survives client disconnectNoYesYes
Who owns resumeThe clientYour serverThe engine
Timers and escalationYou build themYou build themBuilt in
Ops burdenLowMedium (webhook security, retries)Highest (another stateful system)
Best fitDemos, internal toolsMost production agentsHours-long pauses, many steps, audit needs

Durable State and What to Persist

Agent state persistence saves the conversation, tool ledger, and pending intent needed to reconstruct a paused run after a long wait.

Whatever architecture carries the resume, the checkpoint has to hold enough to reconstruct the run. Agent state persistence is not just saving the chat transcript:

  • The conversation so far and a tool-call ledger: which calls completed, with what results, and which idempotency keys were already issued
  • The pending intent: the exact action awaiting approval, its parameters, and the requester
  • A schema version, so newly deployed code can recognize old paused runs
  • Enough identity context to re-authenticate on resume
  • Approval metadata: owner, deadline, escalation chain, timeout policy

Frameworks differ in how much of this they hand you. LangGraph's interrupt support checkpoints the graph at the pause and resumes from the interruption. Google's ADK documents session state that persists between turns, which is what the 83-second walkthrough leans on. With a raw model SDK, persistence is entirely yours; OpenAI's conversation state guide is blunt that you must store and resend the history yourself. Whichever you use, agent checkpointing is only as good as its version field.

Two distinct bug classes live here, and teams conflate them. Losing state entirely is the visible one: the agent resumes blank and the user repeats themselves. Replaying intact state against a world that moved is the subtle one: the approval took nine hours, the invoice was already paid by hand, and a faithful replay of yesterday's plan is still wrong. Because deploying new agent code while old runs sit paused is normal operation rather than an exception, persisted state needs versioning or migration, or resume will deserialize shapes your new code cannot read.

What Breaks Across an Hours-Long Pause

These failure modes are invisible in demos because demos never pause long enough to hit them, and they only strike human-in-the-loop AI agents once real approval latency enters the picture.

  • Replayed side effects. Any resume architecture can execute a side-effecting step at least once more than intended, through a timeout, a retry, or a duplicate webhook delivery. Idempotency keys for agent tool calls are mandatory rather than defensive polish. Derive a stable key from intent id plus step name, send it with the call, and make the tool reject replays; the IETF's Idempotency-Key header draft and Stripe's API are the reference patterns.
  • Expiry on three clocks. Sessions, access tokens, and model context each expire on different schedules, and access tokens commonly live an hour or less. Session expiry in long-running AI agents bites exactly at resume time, so the agent must re-authenticate and rehydrate context before its first tool call rather than assume the original session survived.
  • Stale inputs. The approval answered yesterday's question. Re-fetch the record at resume, diff it against what the approver actually saw, and surface the change instead of executing against a moved world.
  • Duplicate and conflicting approvals. Two approvers answer in opposite order. Enforce one decision per intent id, first write wins, and log both attempts.
  • Deploy drift. New code, old checkpoints. Test resume across a deployment before launch, not during your first incident.

Designing the Approval Handoff

The approval moment is the one part of human-in-the-loop AI agents the user actually sees, so treat it as product surface.

Choose the channel where the approver already lives. A Slack or Teams prompt beats a dashboard nobody opens. A ticket with a due date beats a notification that scrolls away.

Give the approver enough context to decide. Proposed action, key inputs, cost or blast radius, and a link to the full trace. An approver who cannot see what the agent is about to do either rubber-stamps or ignores, and both are oversight failures.

Write the approval timeout policy down. A pending approval needs an explicit deadline and one of three defensible defaults:

Deny on timeout (the safe default for money and access), escalate to the next human in the chain, or auto-approve with audit for low-risk, high-volume actions. Silent indefinite waiting is the one indefensible default: the run leaks, the requester refreshes forever, and nobody owns the decision.

Keep the audit trail. Who decided, when, on what evidence, and what the agent did next. Regulators and incident reviews will ask for exactly this, and NIST's human oversight appendix is a reasonable frame for what good oversight evidence looks like.

A Shipping Checklist for Long-Running Agents

  1. Classify the workload: p95 wait, blocking cost, retry blast radius.
  2. Pick the resume architecture explicitly and record why: polling for internal tools, webhook plus 202 for most production, durable execution for hours-long or many-step runs.
  3. Assign an idempotency key to every mutating tool call and test the replay path.
  4. Checkpoint conversation, tool ledger, pending intent, schema version, identity, and approval metadata.
  5. Version persisted state and test resume across a deploy.
  6. Re-authenticate and rehydrate context on resume; assume every session, token, and context window is stale.
  7. Revalidate inputs at resume time and surface diffs instead of executing blindly.
  8. Enforce one decision per intent; handle duplicate approvals and webhook redelivery.
  9. Define the timeout policy, deadline, and escalation chain before launch.
  10. Emit an audit trail of every approval, decision, and resulting action.

Then map your tools onto the rule: sub-minute internal waits stay synchronous; minutes in production means webhook plus 202; hours, many steps, or audit obligations means a durable execution engine. Step Functions task tokens, LangGraph interrupts, and ADK session state all reduce to one of these three shapes.

The 83-second demo proves an agent can wait. Production asks whether it can come back hours later, act exactly once, and explain itself. That is resume-path engineering, and it is now on your checklist.

Stay in the loop.

Get the latest posts and exclusive content delivered to your inbox.

Join 13 readers. No spam. Unsubscribe in one click, anytime.

About the author

David Moreno

Applied AI Strategist

David helps teams put AI to work in real businesses. He writes teardowns of how companies actually deploy models: the architectures, the trade-offs, and the results that survive contact with the real world.

Related Posts