DiffusionGemma Makes Full Response Latency the Metric
DiffusionGemma teardown: parallel decoding trades first-token latency for full-response latency, drops the KV cache, and breaks streaming and tool calls.
Tag
Posts tagged with diffusion-models
2 posts
DiffusionGemma teardown: parallel decoding trades first-token latency for full-response latency, drops the KV cache, and breaks streaming and tool calls.
Training a diffusion model from scratch can cost hundreds, not millions. The full math behind a 210M DiT built in 3.5 days on one RTX PRO 6000.