---
title: "The Annotated Diffusion Model"
description: "Denoising Diffusion Probabilistic Models generate data samples by learning to reverse a discrete-time forward diffusion process that incrementally adds Gaussian noise to inputs. The forward process adds noise across a predefined schedule of variance parameters up to a fixed number of steps, transforming data into an isotropic Gaussian distribution. Because the true reverse conditional probability distribution is intractable, a neural network optimizes the variational lower bound to approximate the distribution mean at each step while keeping the variance fixed. Sampling subsequently proceeds by iteratively denoising pure Gaussian noise through successive network evaluations to reconstruct a clean data sample. Although diffusion models demonstrate high generation quality, their primary drawback remains the requirement of multiple forward passes during inference compared to models like generative adversarial networks."
---

# The Annotated Diffusion Model

huggingface.co · Niels Rogge, Kashif Rasul · Jun 7, 2022

**Type:** Tutorial

## Summary

Denoising Diffusion Probabilistic Models generate data samples by learning to reverse a discrete-time forward diffusion process that incrementally adds Gaussian noise to inputs. The forward process adds noise across a predefined schedule of variance parameters up to a fixed number of steps, transforming data into an isotropic Gaussian distribution. Because the true reverse conditional probability distribution is intractable, a neural network optimizes the variational lower bound to approximate the distribution mean at each step while keeping the variance fixed. Sampling subsequently proceeds by iteratively denoising pure Gaussian noise through successive network evaluations to reconstruct a clean data sample. Although diffusion models demonstrate high generation quality, their primary drawback remains the requirement of multiple forward passes during inference compared to models like generative adversarial networks.

## Context

Generative modeling requires converting noise into real data samples, but directly computing the intractable true conditional probability distribution needed to reverse noise degradation across arbitrary image distributions is impossible.

## Approach / What changed

A discrete-time forward diffusion process incrementally injects Gaussian noise across defined variance schedules, while a PyTorch neural network is trained using the evidence lower bound to learn the conditional mean of the reverse denoising process.

## Takeaways

- In the original DDPM implementation, the variance of the reverse process is kept fixed to untrained time-dependent constants while the neural network only learns the conditional mean.
- The training objective is derived from the variational lower bound (ELBO), where intermediate terms reduce to KL divergences between Gaussians parameterized as L2 losses.
- A primary computational drawback of diffusion models relative to GANs is the requirement of repeated neural network forward passes to iteratively sample an image from noise.

**Tags:** [Machine Learning](https://yomu.fyi/topic/machine-learning), [Python](https://yomu.fyi/topic/python)

- Source: [huggingface.co](https://huggingface.co/blog/annotated-diffusion)
- Source URL: https://huggingface.co/blog/annotated-diffusion
- Ingested by Yomu: 2026-08-27T15:10:25.435Z

[Read original post](https://huggingface.co/blog/annotated-diffusion)