---
title: "Multi-Token Prediction"
description: "1 posts about Multi-Token Prediction, summarised, each linking to the original."
---

# Multi-Token Prediction
> 1 posts about Multi-Token Prediction, summarised, each linking to the original.

## Articles

### [Multi-token Residual Prediction](https://yomu.fyi/post/multi-token-residual-prediction.md)
- Company: [Modal](https://yomu.fyi/company/modal.md)
- Published: Jul 1, 2026
- Type: Explainer

Multi-Token Residual Prediction (MRP) adapts multi-token prediction for diffusion language models by training a small module to predict the residual between adjacent denoising steps rather than the full next-step distribution. This residual target is easier to learn, allowing a tiny 3-layer transformer to accurately predict corrections across multiple steps. In the static regime, MRP enables lossless speculative decoding with up to 1.56× throughput in SGLang, or direct decoding with a small quality cost. In the dynamic regime, MRP recovers up to +16 accuracy points lost by aggressive low-threshold decoding through a remasking mechanism. The module is trained on a frozen backbone and composes cleanly with existing inference methods.