FEATUREDComputing Life · Share (鸭哥 research reports)· rssZH00:00 · 08·16
→Google open-sources DiffusionGemma: a diffusion-based Gemma 4 hitting 1,456 tok/s decode, with a clear reasoning trade-off
Google converted the fully post-trained Gemma 4 26B-A4B weights into a discrete polynomial diffusion model and open-sourced the weights on Hugging Face. On a single H100 at FP8 with batch size 1, decode hits 1,456 tok/s—over 7× the original AR model—by processing 256 tokens per forward pass and cutting memory-bandwidth overhead at low concurrency. The trade-off: AIME 2026 drops from 88.3 to 69.1, and MRCR 128K from 44.1 to 32.0. An AR fallback mode recovers AIME to 84.2, showing the base knowledge survived but the diffusion generation mode itself caused part of the quality loss. Additional training used under 10% of the original token budget, but absolute token count, FLOPs, and GPU hours are not disclosed. In real serving, TTFT rises from 53 ms to 489 ms, and at high concurrency AR total throughput overtakes diffusion.
#Reasoning#Agent#Code#Google
why featured
Featured · importance 82 · hook + knowledge + resonance
editor take
Google open-sourced DiffusionGemma, a diffusion-converted Gemma 4 26B hitting 1,456 tok/s on one H100, but AIME 2026 drops from 88.3 to 69.1.
sharp
This one's worth opening because Google didn't train a diffusion model from scratch—they converted the fully post-trained Gemma 4 26B-A4B weights directly, and open-sourced the result. The approach is efficient: adaptation training plus sampler distillation used under 10% of the original training token budget, letting the model denoise a 256-token canvas in parallel. On a single H100 at FP8 with batch size 1, pure decode hits 1,456 tok/s, over 7× the original AR model.
The speedup comes from a real hardware bottleneck. At low concurrency, AR models waste most of their time reading weights and KV cache from memory for each single-token forward pass. Diffusion processes 256 positions per forward pass, cutting the number of serial rounds and the memory-bandwidth overhead.
The trade-off shows up in math. AIME 2026 drops from 88.3 to 69.1, and MRCR 128K from 44.1 to 32.0. The key ablation: switching to AR fallback mode recovers AIME to 84.2, so the base knowledge survived—the diffusion generation step itself caused the quality loss.
Two deployment catches. TTFT jumps from 53 ms to 489 ms, which users will feel. And at high concurrency, AR total throughput overtakes diffusion, so this is best for low-concurrency, long-output workloads where per-token latency matters less.
Don't read this as "diffusion replaces autoregressive." The more accurate take: Google shipped a usable diffusion inference option with open weights and vLLM support that engineering teams can test today. But the paper doesn't disclose absolute token count, FLOPs, or GPU hours for the adaptation phase, so the full cost picture is still incomplete.
HKR breakdown
hook ✓knowledge ✓resonance ✓