---
title: "Boost Inference Performance up to 15x on NVIDIA Blackwell Using DFlash Speculative Decoding"
description: "Sequential token generation in autoregressive LLMs limits GPU utilization and serving throughput, while conventional speculative decoding remains constrained by autoregressive draft models. DFlash addresses this bottleneck by employing a lightweight block-diffusion drafter that predicts an entire block of masked candidate tokens in a single forward pass. The architecture incorporates target hidden-state conditioning and key-value injection into draft model projections while relying on the target model for parallel verification. Evaluated on NVIDIA Blackwell GPUs across TensorRT-LLM, vLLM, and SGLang, DFlash increases gpt-oss-120b throughput by more than 15x at high interactivity targets and delivers up to 5.8x speedups on Gemma 4 31B compared to autoregressive decoding."
---

# Boost Inference Performance up to 15x on NVIDIA Blackwell Using DFlash Speculative Decoding

[NVIDIA Developer Blog](https://yomu.fyi/company/nvidia-developer-blog) · Amr Elmeleegy · Jun 23, 2026

**Type:** Benchmark

## Summary

Sequential token generation in autoregressive LLMs limits GPU utilization and serving throughput, while conventional speculative decoding remains constrained by autoregressive draft models. DFlash addresses this bottleneck by employing a lightweight block-diffusion drafter that predicts an entire block of masked candidate tokens in a single forward pass. The architecture incorporates target hidden-state conditioning and key-value injection into draft model projections while relying on the target model for parallel verification. Evaluated on NVIDIA Blackwell GPUs across TensorRT-LLM, vLLM, and SGLang, DFlash increases gpt-oss-120b throughput by more than 15x at high interactivity targets and delivers up to 5.8x speedups on Gemma 4 31B compared to autoregressive decoding.

## Context

Autoregressive LLMs generate tokens sequentially, which limits GPU utilization and constrains throughput in latency-sensitive serving scenarios. While speculative decoding mitigates this by using a lightweight model to draft tokens for parallel verification by a target model, traditional speculative methods rely on autoregressive draft models that still draft sequentially. Consequently, drafting costs grow with the speculative token count, creating a bottleneck on modern high-compute GPU architectures.

## Approach / What changed

DFlash implements block-diffusion speculative decoding by replacing sequential draft models with a lightweight drafter that predicts a block of masked future tokens in a single forward pass. The drafter conditions on target model hidden states and injects target context features into its key-value projections across layers to preserve high acceptance rates. The target model then verifies candidate blocks in parallel, maintaining output distribution fidelity across serving frameworks like TensorRT-LLM, vLLM, and SGLang.

## Takeaways

- DFlash replaces sequential drafting with single-pass block-diffusion drafting, combining target hidden-state conditioning and multi-layer KV injection while preserving target model output distributions.
- On an eight NVIDIA DGX B300 system running TensorRT-LLM, DFlash delivers over 15x higher throughput for gpt-oss-120b than autoregressive decoding at 500 to 600 tokens/sec per user, beating EAGLE-3 by 1.5x.
- DFlash integrates into vLLM via the Speculators library and into SGLang through configuration updates, achieving up to 5.8x speedup on Gemma 4 31B and 5.1x on Qwen3 8-B on single Blackwell GPUs.

**Tags:** [LLMs](https://yomu.fyi/topic/llm), [Machine Learning](https://yomu.fyi/topic/machine-learning), [Open Source](https://yomu.fyi/topic/open-source), [Performance](https://yomu.fyi/topic/performance)

- Source: [NVIDIA Developer Blog](https://developer.nvidia.com/blog/boost-inference-performance-up-to-15x-on-nvidia-blackwell-using-dflash-speculative-decoding)
- Source URL: https://developer.nvidia.com/blog/boost-inference-performance-up-to-15x-on-nvidia-blackwell-using-dflash-speculative-decoding
- Ingested by Yomu: 2026-08-27T14:58:08.283Z

[Read original post](https://developer.nvidia.com/blog/boost-inference-performance-up-to-15x-on-nvidia-blackwell-using-dflash-speculative-decoding)