Tag

#latency

Posts tagged with latency

2 posts

DiffusionGemma builds entire responses through parallel denoising instead of one token at a time, making full-response latency the metric that matters for serving economics.

Engineering • Nov 06, 2026 •11 min read

DiffusionGemma Makes Full Response Latency the Metric

DiffusionGemma teardown: parallel decoding trades first-token latency for full-response latency, drops the KV cache, and breaks streaming and tool calls.

#diffusiongemma #diffusion-models +3

Gemini 3.8 Live extended thinking lets a realtime voice agent reason before speaking, and that reasoning pause must fit inside a strict end-to-end latency budget.

Tools • Sep 17, 2026 •10 min read

Gemini 3.8 Live Extended Thinking Needs a Latency Budget

Gemini 3.8 Live extended thinking adds reasoning time to a realtime voice agent. The latency-budget math tells you when it helps and when it is dead air.

#gemini-38 #extended-thinking +3