What you’ll learn

• the forward diffusion process: gradually destroy an image by adding a little Gaussian noise at each of many steps, until it becomes pure noise
• the reverse diffusion process: a network trained to undo one small noise step at a time, walking backward from pure noise to a realistic image
• the DDPM training objective — predict the noise, not the image
• why diffusion models train far more stably than GANs, at the cost of slow, multi-step sampling

Melting an image down, then rebuilding it

Autoencoders squeeze data through a bottleneck, and GANs pit two networks against each other. Diffusion models take a completely different route: they learn to reverse a process of decay.

Imagine slowly dissolving a photograph into television static — a tiny bit of Gaussian noise added at each of hundreds of steps, until nothing recognizable is left. That’s the forward process, and it needs no learning at all: it’s just a fixed noise schedule you apply directly to your training images. The interesting part is the reverse process: a neural network trained to look at a noisy image and guess what noise was just added, so that subtracting its guess nudges the image one step back toward something clean. Chain enough of those small denoising steps together, starting from pure noise, and you can walk all the way back to a brand-new, realistic image that was never in the training set.

  flowchart LR
  X0["x0: clean image"] -->|"+ noise"| X1["x1"]
  X1 -->|"+ noise"| X2["... xT-1"]
  X2 -->|"+ noise"| XT["xT: pure noise"]
  XT -->|"predict noise, subtract"| R2["... x1"]
  R2 -->|"predict noise, subtract"| R1["x0': generated image"]

The forward process, in closed form

The forward process adds noise according to a variance schedule beta_tbeta_t that grows slightly at each of TT steps. The elegant part: because Gaussians combine nicely, you don’t have to simulate all the intermediate steps one by one — there’s a closed-form shortcut straight from the clean image x0x0 to any noisy step xtxt:

xt = sqrt(alpha_bar_t) * x0 + sqrt(1 - alpha_bar_t) * noise

xt = sqrt(alpha_bar_t) * x0 + sqrt(1 - alpha_bar_t) * noise

where alpha_bar_talpha_bar_t is the running product of (1 - beta_s)(1 - beta_s) for every step up to tt. As tt grows, alpha_bar_talpha_bar_t shrinks toward 0 — meaning less and less of the original signal survives, and more and more of the output is pure noise:

import numpy as np
 
T = 200
betas = np.linspace(1e-4, 0.02, T)       # variance schedule: small steps at first
alphas = 1 - betas
alpha_bars = np.cumprod(alphas)          # running product = closed-form shortcut
 
print("alpha_bar at t=0:  ", round(alpha_bars[0], 4))
print("alpha_bar at t=100:", round(alpha_bars[99], 4))
print("alpha_bar at t=200:", round(alpha_bars[199], 4))
print("signal fraction retained at t=200:", round(np.sqrt(alpha_bars[199]), 4))
print("noise fraction added at t=200:    ", round(np.sqrt(1 - alpha_bars[199]), 4))

import numpy as np
 
T = 200
betas = np.linspace(1e-4, 0.02, T)       # variance schedule: small steps at first
alphas = 1 - betas
alpha_bars = np.cumprod(alphas)          # running product = closed-form shortcut
 
print("alpha_bar at t=0:  ", round(alpha_bars[0], 4))
print("alpha_bar at t=100:", round(alpha_bars[99], 4))
print("alpha_bar at t=200:", round(alpha_bars[199], 4))
print("signal fraction retained at t=200:", round(np.sqrt(alpha_bars[199]), 4))
print("noise fraction added at t=200:    ", round(np.sqrt(1 - alpha_bars[199]), 4))

alpha_bar at t=0:   0.9999
alpha_bar at t=100: 0.6025
alpha_bar at t=200: 0.1322
signal fraction retained at t=200: 0.3636
noise fraction added at t=200:     0.9316

alpha_bar at t=0:   0.9999
alpha_bar at t=100: 0.6025
alpha_bar at t=200: 0.1322
signal fraction retained at t=200: 0.3636
noise fraction added at t=200:     0.9316

By t=200t=200, less than 40% of the original signal is left, and the rest is noise — after a few hundred more steps with a real schedule, essentially nothing of x0x0 survives.

The reverse process: predict the noise, not the image

Training a diffusion model doesn’t require running the forward process step by step either. For each training image, you pick a random timestep tt, jump straight to the noisy xtxt using the formula above, and ask a network to predict exactly which noisenoise was added. The loss (from the DDPM paper) is refreshingly simple — just mean squared error between the real and predicted noise:

import numpy as np
import tensorflow as tf
 
np.random.seed(0)
tf.random.set_seed(0)
 
# Toy dataset: 200 tiny 3-value "images", noised at a single fixed step (alpha_bar = 0.36)
X0 = np.random.uniform(-1, 1, size=(200, 3)).astype("float32")
noise = np.random.normal(size=X0.shape).astype("float32")
Xt = (0.6 * X0 + 0.8 * noise).astype("float32")
 
# The network's ONLY job: given a noisy sample, predict the noise that was added
noise_predictor = tf.keras.models.Sequential([
    tf.keras.layers.Dense(16, activation="relu", input_shape=[3]),
    tf.keras.layers.Dense(3),
])
noise_predictor.compile(loss="mse", optimizer="adam")
history = noise_predictor.fit(Xt, noise, epochs=5, verbose=0)
 
print("loss decreased:", history.history["loss"][-1] < history.history["loss"][0])

import numpy as np
import tensorflow as tf
 
np.random.seed(0)
tf.random.set_seed(0)
 
# Toy dataset: 200 tiny 3-value "images", noised at a single fixed step (alpha_bar = 0.36)
X0 = np.random.uniform(-1, 1, size=(200, 3)).astype("float32")
noise = np.random.normal(size=X0.shape).astype("float32")
Xt = (0.6 * X0 + 0.8 * noise).astype("float32")
 
# The network's ONLY job: given a noisy sample, predict the noise that was added
noise_predictor = tf.keras.models.Sequential([
    tf.keras.layers.Dense(16, activation="relu", input_shape=[3]),
    tf.keras.layers.Dense(3),
])
noise_predictor.compile(loss="mse", optimizer="adam")
history = noise_predictor.fit(Xt, noise, epochs=5, verbose=0)
 
print("loss decreased:", history.history["loss"][-1] < history.history["loss"][0])

loss decreased: True

loss decreased: True

A real diffusion model also feeds the timestep tt into the network (so it knows how noisy the input is) and uses a U-Net instead of a couple of DenseDense layers — but the training objective is exactly this: predict the noise, compare it to the real noise, backpropagate.

Sampling — actually generating an image — runs this in reverse: start from pure Gaussian noise xTxT, repeatedly ask the network “what noise is in here?”, subtract a (small, carefully scaled) piece of its prediction, and repeat for every one of the TT steps back down to x0x0. That’s also the main downside compared to a GAN or a VAE: generating one image takes many sequential network passes instead of one.

Why diffusion over GANs?

GAN training pits two networks against each other in a minimax game that can diverge or mode-collapse (see the previous page). A diffusion model’s training target is just “predict this noise” — an ordinary supervised regression problem with no adversary and no competing objective, so training tends to be far more stable and doesn’t require the delicate hyperparameter balancing GANs need. The trade-off is speed: a GAN generates an image in a single forward pass, while a diffusion model needs dozens to thousands of sequential denoising steps. This stability-versus-speed trade-off is exactly why diffusion models (DDPMs, and their descendants like Stable Diffusion) overtook GANs as the state of the art for high-quality image generation.

Visualize it

Watch a small pattern get buried under noise, step by step (forward diffusion), and then walk back out of the noise to the original pattern (reverse diffusion) — the same trip, run in both directions:

Mini-checkpoint

If a GAN generates an image in one forward pass, why would anyone put up with a diffusion model’s much slower, multi-step sampling?

• Because GAN training is unstable and prone to mode collapse, while a diffusion model’s training objective is a plain regression problem (predict the noise) with no adversary — it converges more reliably and tends to produce more diverse, higher-quality samples. Slower sampling is the price paid for a much easier, steadier training process.

🧪 Try It Yourself

Exercise 1 – Build the Variance Schedule

Exercise 2 – Jump Straight to a Noisy Step

Exercise 3 – Train a Tiny Noise Predictor

Next

Continue to Text Generation with Language Models — a completely different generative task (sequences instead of images) with its own key idea: the softmax sampling temperature.