Gemini 3.8 Live Extended Thinking Needs a Latency Budget

Google's launch materials present Gemini 3.8 Live extended thinking as smarter conversation, with the model pausing to reason before it speaks. That framing omits the engineering cost. In a realtime voice agent, thinking is the newest line item in an end-to-end latency budget that already contains endpointing, transcription or native audio processing, prefill, decode, and synthesis, not a free quality knob you turn up. Human conversation runs on turn gaps averaging roughly 200 ms, which means most voice turns can absorb only a modest thinking budget before reasoning stops buying accuracy and starts buying dead air. This piece does the arithmetic the launch recap will not: a fill-in-the-numbers latency budget, three worked scenarios, the thinking-token cost per conversation minute, and the interruption regressions that appear the day you enable it.

What the Gemini 3.8 Live extended thinking launch says

Per the Gemini 3.8 audio model card and the launch post, the 3.8 Live models run realtime audio in and audio out, and the Extended Thinking variant lets the model reason internally before responding, with a budget developers configure through the Live API. Google's own evaluations report gains on multi-step tasks.

What the materials do not publish, as of this writing, matters more for builders: per-turn first-audio latency deltas for live sessions, thinking-token consumption per minute of representative conversation, and measured behavior when a user barges in mid-reasoning. Treat launch-week demo smoothness as unverified until your own session logs reproduce it.

The contrast with the text stack explains why this launch is different. In a chat completion, thinking merely lengthens a response that readers consume whenever it arrives. In a Live session, the reasoning phase gates first audio, so its cost is denominated in conversational silence, the one resource voice UX cannot manufacture more of.

The end-to-end latency budget of a realtime voice agent

Three numbers bound voice AI response time from the outside, and all three predate LLMs. A PNAS turn-taking study (Stivers et al., 2009) found that people hand off conversational turns with gaps averaging around 200 ms, with striking consistency across the ten languages sampled, so sub-second response is the rhythm listeners grew up with, not a nice-to-have. The ITU-T G.114 recommendation treats roughly 400 ms as the ceiling for acceptable one-way mouth-to-ear delay in telephony; it was written for network transport, so use it as an anchor for total delay rather than a strict rule. And the NN/g response-time limits place the boundary for holding a user's flow at about one second. How much silence users tolerate in voice agents is set by those inherited expectations: acknowledge inside roughly 1 second, and remember the listener's internal clock started at their last word.

Realtime voice agent latency is the sum of the components below. These are typical planning ranges, not benchmarks; substitute your own p95 measurements.

Sum the midpoints and a competent stack lands between roughly 0.7 seconds (native audio, tight endpointing) and 1.8 seconds (full ASR-to-LLM-to-TTS cascade) before any thinking exists. Many pipelines are already at or past the 1 second line. Speech-to-speech latency budgets were tight before reasoning arrived, and Gemini Live extended thinking draws from the same account. In a Live session, thinking time is additive to the rest of the pipeline, so all else equal your first spoken response shifts by roughly the thinking duration. Nothing overlaps to hide it.

How to compute your Live API thinking budget

How to compute a voice agent latency budget in one line: available thinking time equals your perceived-response target, minus the rest of the pipeline, minus a safety margin. Measure the pipeline at p95 rather than p50, because tails are what callers remember. Pick the target by product: roughly 0.8 to 1.0 seconds for consumer companion apps, 1.0 to 1.2 seconds for support lines, and only push beyond that for internal tools where users are paid to wait.

Three worked scenarios follow. All numbers are illustrative planning assumptions; replace them with your measured p95s.

A native audio pipeline that fits

Native speech-to-speech over the Live API, the shape Google recommends: endpointing 250 ms, prefill plus first-audio decode 450 ms, transport 80 ms. Base cost 780 ms. Against a 1.1 second target with a 120 ms margin, you hold about 200 ms of thinking headroom. That happens to sit at the human turn-gap rhythm, so light reasoning lands as a natural beat rather than a stall.

A cascade stack that barely fits

ASR feeding a text model feeding TTS: endpointing 250, transcription 200, prefill 300, decode 300, synthesis 150, transport 120. Base cost 1,320 ms, already 320 ms past a 1 second target before thinking exists. Even a forgiving 1.6 second target leaves 280 ms, and a sane margin consumes 150 of them. The thinking budget is roughly 130 ms, which is noise. Fix the stack or move to native audio before paying for reasoning.

A tool-orchestration turn that breaks

A booking turn that needs four seconds of reasoning across calendar and inventory tools: 1,320 ms base plus 4,000 ms of thinking is about 5.3 seconds of pure silence, five times the ceiling. The arithmetic says this cannot ship as silence. It can ship with acknowledge-then-answer, an immediate spoken acknowledgment that restarts the user's patience clock, with reasoning and tool calls executing while the acknowledgment plays.

What extended thinking buys, by workload

When extended thinking helps a realtime voice agent comes down to turn type, and Gemini 3.8 Live extended thinking follows the same test-time-compute economics as any text model: reasoning tends to pay where an answer must be searched, checked, or computed, and tends to waste tokens on lookup and confirmation.

The gains that justify a pause concentrate in multi-constraint, arithmetic, and tool-orchestration turns, while confirmation and slot-filling turns often show little measurable benefit, so budget spent there buys pure latency and pure token cost. Note the asymmetry: the turns that benefit most are usually the rarest in a conversation, which argues for per-turn allocation rather than a session-wide default.

Cost per conversation minute with thinking enabled

Under Gemini API pricing, thinking tokens bill at the same rate as output tokens, so reasoning is never a rounding error on the invoice. Run the arithmetic with stated assumptions: a dense support conversation at six user turns per minute, averaging 1,500 thinking tokens per turn. That is 9,000 billed reasoning tokens per conversation minute, or 540,000 per hour. Compare spoken output at roughly three tokens per second while the model talks, perhaps 30 seconds of speech per minute, around 90 to 120 tokens. Reasoning outweighs speech by roughly 75 to 100 times in billed volume, and at any plausible output price that makes thinking the dominant driver of cost per conversation minute.

Halve the assumptions and the conclusion survives: 500 thinking tokens per turn at four turns per minute is still 2,000 tokens against a hundred or so spoken, an order of magnitude. Two more cost edges to watch: aborted reasoning after a barge-in may still bill for tokens generated before cancellation, and long conversations compound the per-minute delta in exactly the sessions where retention matters most. Budget in both tokens and seconds, log thinking tokens per turn class, and treat cost per conversation minute as a first-class metric next to p95 latency.

Why thinking complicates interruptions and turn-taking

Dialogue research on barge-in, including Disney Research barge-in work, has long shown that people interrupt as soon as they can predict where a turn is going. Extended thinking opens a new window where that happens: a user who misspoke will barge in during the reasoning phase itself. Endpointing fires, and per the interruption handling covered in the Live API best-practices guide, the session then has to cancel or discard the in-flight reasoning, because it was premised on an utterance the user just revised.

Interruption handling with thinking enabled in the Live API is a redesign, not a config flag. Three things change. Aggressive endpointing plus thinking means false interruptions now abort expensive reasoning. Every abort may bill partial thinking tokens and restart reasoning from zero on the next turn, a double penalty. And because listeners bring that 200 ms rhythm, a silent thinking pause reads as the agent not listening, since nothing signals work is happening.

Mitigations that preserve perceived responsiveness

• Acknowledge then answer. Speak a short acknowledgment inside your latency budget, then think while it plays.
• Use fillers for long budgets. A filler-phrase perception study from 2025 found that even brief, task-relevant fillers can hold up perceived responsiveness while backend work completes, exactly the property a thinking pause needs to borrow.
• Set dynamic per-turn budgets: near zero for confirmations, capped for tool turns.
• Gate barge-in sensitivity tighter during acknowledgment playback and looser during silence.
• Track barge-in-during-thinking rate as a regression metric, not an afterthought.

A decision framework for enabling extended thinking

The pattern that survives contact with the arithmetic is a per-turn thinking budget with dynamic tiering, near zero for confirmations and capped for tool calls, not a global on or off switch. A short configuration checklist:

1. Set budgets in tokens and seconds, per turn class.
2. Instrument p50 and p95 time-to-first-audio per turn class.
3. Add a silence gate at about 1 second that falls back to low-thinking mode.
4. Route any turn with a budget above ~1 second through an acknowledgment.
5. Log barge-in rate during thinking phases and thinking tokens per turn.
6. A/B answer quality against latency before concluding thinking helps; the launch demo is not evidence for your workload.

The verdict for builders

Enable Gemini 3.8 Live extended thinking now if your hard turns are multi-step and you can acknowledgment-fill them, because the arithmetic shows native-audio pipelines carry a few hundred milliseconds of headroom and tool turns can hide reasoning behind speech. Skip it if your turns are short and frequent, because the same arithmetic says reasoning buys nothing while charging silence and tokens. Most Gemini 3.8 Live review coverage will grade the model on benchmarks it picked for itself. Builders get graded on p95 time-to-first-audio, barge-in regressions, and cost per conversation minute, and on those three metrics the decision is arithmetic, not sentiment. Budget per turn, tier by workload, and keep the toggle off by default until your own numbers say otherwise.