---
title: "Introducing Waypoint-1: Real-time interactive video diffusion from Overworld"
description: "Overworld introduced Waypoint-1, a real-time interactive video diffusion model controllable through text prompts, mouse movements, and keyboard inputs. The architecture uses a frame-causal rectified flow transformer trained on compressed latent representations across 10,000 hours of video game footage paired with control signals. Pre-training uses diffusion forcing with causal attention, while post-training incorporates self forcing via DMD to mitigate autoregressive rollout error accumulation, enable one-pass classifier-free guidance, and support few-step denoising. To achieve low latency on consumer hardware, Overworld developed WorldEngine, a pure Python streaming inference library. Utilizing AdaLN feature caching, static rolling key-value caching with Flex Attention, fused matrix multiplications, and PyTorch compilation, the 2.3B parameter Waypoint-1-Small model reaches up to 60 frames per second on an RTX 5090 GPU."
---

# Introducing Waypoint-1: Real-time interactive video diffusion from Overworld

[Hugging Face](https://yomu.fyi/company/hugging-face) · Andrew Lapp, Louis Castricato, Scott Fox, Shahbuland Matiana, David Rossi · Jan 20, 2026

**Type:** Announcement

## Summary

Overworld introduced Waypoint-1, a real-time interactive video diffusion model controllable through text prompts, mouse movements, and keyboard inputs. The architecture uses a frame-causal rectified flow transformer trained on compressed latent representations across 10,000 hours of video game footage paired with control signals. Pre-training uses diffusion forcing with causal attention, while post-training incorporates self forcing via DMD to mitigate autoregressive rollout error accumulation, enable one-pass classifier-free guidance, and support few-step denoising. To achieve low latency on consumer hardware, Overworld developed WorldEngine, a pure Python streaming inference library. Utilizing AdaLN feature caching, static rolling key-value caching with Flex Attention, fused matrix multiplications, and PyTorch compilation, the 2.3B parameter Waypoint-1-Small model reaches up to 60 frames per second on an RTX 5090 GPU.

## Context

Existing world models typically fine-tune pre-trained video models using simplified, sparse control inputs, resulting in camera rotation constraints and severe latency issues. Additionally, training models purely with standard diffusion forcing creates an inference mismatch during autoregressive rollouts, which leads to error accumulation and noisy streams.

## Approach / What changed

Overworld developed Waypoint-1, training a frame-causal rectified flow transformer on 10,000 hours of video game footage with continuous keyboard and mouse inputs. The model uses diffusion forcing for pre-training and self forcing via DMD for post-training, executed via the WorldEngine Python inference library optimized with AdaLN caching, rolling KV caching, matmul fusion, and Torch Compile.

## Takeaways

- Waypoint-1 is a 2.3B latent frame-causal rectified flow transformer trained on 10,000 hours of video game footage with text captions and keyboard and mouse inputs.
- Post-training with self forcing via DMD mitigates autoregressive rollout error accumulation while enabling one-pass classifier-free guidance and few-step denoising.
- The WorldEngine Python inference library achieves 30 FPS at 4 steps and 60 FPS at 2 steps for Waypoint-1-Small on an RTX 5090 using AdaLN caching, static rolling KV cache, fused QKV, and Torch Compile.

**Tags:** [Caching](https://yomu.fyi/topic/caching), [Machine Learning](https://yomu.fyi/topic/machine-learning), [Performance](https://yomu.fyi/topic/performance), [Python](https://yomu.fyi/topic/python), [Streaming](https://yomu.fyi/topic/streaming)

- Source: [Hugging Face](https://huggingface.co/blog/waypoint-1)
- Source URL: https://huggingface.co/blog/waypoint-1
- Ingested by Yomu: 2026-08-27T20:00:53.513Z

[Read original post](https://huggingface.co/blog/waypoint-1)