Google has highlighted a UCSD-led port of DFlash to its TPU inference stack, pointing to one of the clearer recent signs that speculative decoding research is starting to matter in real serving systems.

What changed

According to Google’s developers blog, the UCSD team integrated DFlash, a block-diffusion speculative decoding method, into the open-source vLLM TPU ecosystem and measured an average 3.13x increase in tokens per second on TPU v5p. Google also said some math-heavy benchmarks approached 6x speedups, while a head-to-head comparison against EAGLE-3 on the same TPU generation showed a 2.29x end-to-end serving gain for DFlash versus 1.30x for EAGLE-3.

Those figures line up with the underlying DFlash paper, which describes a draft model that predicts an entire token block in a single forward pass instead of drafting tokens one by one. That parallel approach is the core reason the method can outperform standard autoregressive speculative decoding.

Why it matters

The conservative read is that this is still an engineering result, not a broad production rollout. The vLLM TPU project’s own docs still list v5p as experimental, which suggests developers should treat these numbers as promising but not yet universal. Even so, the result matters for teams working on LLM serving economics: if block-parallel drafting holds up outside benchmarks, it could materially reduce latency on open inference stacks running on Google TPUs.