Skip to content
Python Central Hub

DeepDream

What you’ll learn

  • how DeepDream runs a pretrained convnet in reverse: instead of adjusting weights to fit an image, it adjusts the image to maximize a layer’s activation
  • why this is just gradient ascent instead of gradient descent — you’re climbing toward more activation, not descending toward less loss
  • what an octave is, and why processing an image at several scales produces richer, more detailed dreams
  • why lower layers give geometric patterns and higher layers give recognizable (dog-eye, bird-feather) shapes

Running a convnet backward

Every convnet you’ve trained so far works one direction: pixels go in, a prediction comes out, and training nudges the weights to make that prediction better. DeepDream asks a different question: what if we freeze the weights, and instead nudge the pixels to make some layer’s activation as large as possible?

That’s it — that’s the whole trick. Pick a pretrained convnet (Chollet’s book uses Inception V3), pick one or more of its layers, and define a loss equal to the mean of the squared activations in those layers. Then run gradient ascent — repeatedly compute the gradient of that loss with respect to the input image, and add a small step in that direction. Each step nudges the image to contain a little more of whatever pattern already excites those layers. Because DeepDream starts from a real photograph instead of random noise, the effect looks like the image’s existing edges and textures are “growing” eyes, feathers, and swirls out of themselves — DeepDream’s convnet was trained on ImageNet, where dogs and birds are hugely overrepresented, so those are the patterns that tend to bubble up.

diagram Gradient ascent on the input image mermaid
Instead of updating weights to reduce a loss, DeepDream updates the input image to increase a layer's activation.

The loss: maximize activation, not minimize error

In a normal classifier, the loss measures error and you push it down. Here, the “loss” measures how strongly a layer is firing and you push it up — so every gradient step is image += learning_rate * gradientimage += learning_rate * gradient, addition instead of subtraction. Chollet’s version sums a weighted combination of several layers so the dream mixes several kinds of pattern at once:

DeepDream loss and gradient ascent step (Keras, conceptual)
import tensorflow as tf
from tensorflow import keras
 
model = keras.applications.inception_v3.InceptionV3(weights="imagenet", include_top=False)
 
layer_settings = {
    "mixed4": 1.0,
    "mixed5": 1.5,
    "mixed6": 2.0,
    "mixed7": 2.5,
}
outputs_dict = dict(
    (layer.name, layer.output) for layer in [model.get_layer(name) for name in layer_settings]
)
feature_extractor = keras.Model(inputs=model.inputs, outputs=outputs_dict)
 
def compute_loss(input_image):
    features = feature_extractor(input_image)
    loss = tf.zeros(shape=())
    for name in features:
        coeff = layer_settings[name]
        activation = features[name]
        # skip border pixels to avoid edge artifacts
        loss += coeff * tf.reduce_mean(tf.square(activation[:, 2:-2, 2:-2, :]))
    return loss
 
@tf.function
def gradient_ascent_step(image, learning_rate):
    with tf.GradientTape() as tape:
        tape.watch(image)
        loss = compute_loss(image)
    grads = tf.math.l2_normalize(tape.gradient(loss, image))
    return loss, image + learning_rate * grads
DeepDream loss and gradient ascent step (Keras, conceptual)
import tensorflow as tf
from tensorflow import keras
 
model = keras.applications.inception_v3.InceptionV3(weights="imagenet", include_top=False)
 
layer_settings = {
    "mixed4": 1.0,
    "mixed5": 1.5,
    "mixed6": 2.0,
    "mixed7": 2.5,
}
outputs_dict = dict(
    (layer.name, layer.output) for layer in [model.get_layer(name) for name in layer_settings]
)
feature_extractor = keras.Model(inputs=model.inputs, outputs=outputs_dict)
 
def compute_loss(input_image):
    features = feature_extractor(input_image)
    loss = tf.zeros(shape=())
    for name in features:
        coeff = layer_settings[name]
        activation = features[name]
        # skip border pixels to avoid edge artifacts
        loss += coeff * tf.reduce_mean(tf.square(activation[:, 2:-2, 2:-2, :]))
    return loss
 
@tf.function
def gradient_ascent_step(image, learning_rate):
    with tf.GradientTape() as tape:
        tape.watch(image)
        loss = compute_loss(image)
    grads = tf.math.l2_normalize(tape.gradient(loss, image))
    return loss, image + learning_rate * grads

Two details matter: tape.watch(image)tape.watch(image) is required because imageimage isn’t a trainable weight — it’s just a tensor, so TensorFlow needs to be told explicitly to track gradients through it. And the gradient is L2-normalized before being applied, exactly like the filter-visualization trick from earlier convnet chapters — this keeps each step a similar size regardless of how large the raw gradient happens to be.

Verifying the mechanism with plain NumPy

You don’t need a real convnet to see gradient ascent working. Swap Inception’s layers for a tiny linear “filter bank” WW, define the same sum-of-squared-activations loss, and the gradient is 2 * Wᵀ(W·image)2 * Wᵀ(W·image) — pure NumPy, no TensorFlow required, and you can watch the loss climb step by step:

Gradient ascent on a toy image (verified with plain NumPy)
import numpy as np
 
np.random.seed(0)
 
n = 8
image = np.random.uniform(-1, 1, size=n)          # toy 1D "image"
W = np.random.uniform(-0.5, 0.5, size=(4, n))     # toy "feature detectors"
 
def compute_loss(img):
    activations = W @ img
    return np.sum(activations ** 2)               # mean/sum of squared activations
 
def compute_grad(img):
    activations = W @ img
    return 2 * (W.T @ activations)                 # gradient of the loss w.r.t. the image
 
lr = 0.05
img = image.copy()
loss_before = compute_loss(img)
 
for step in range(10):
    grad = compute_grad(img)
    grad = grad / (np.sqrt(np.mean(grad ** 2)) + 1e-8)   # L2-normalize, like the book
    img = img + lr * grad                                 # ASCENT: add, don't subtract
 
loss_after = compute_loss(img)
print("loss before:", round(loss_before, 4))
print("loss after 10 ascent steps:", round(loss_after, 4))
print("loss increased:", loss_after > loss_before)
Gradient ascent on a toy image (verified with plain NumPy)
import numpy as np
 
np.random.seed(0)
 
n = 8
image = np.random.uniform(-1, 1, size=n)          # toy 1D "image"
W = np.random.uniform(-0.5, 0.5, size=(4, n))     # toy "feature detectors"
 
def compute_loss(img):
    activations = W @ img
    return np.sum(activations ** 2)               # mean/sum of squared activations
 
def compute_grad(img):
    activations = W @ img
    return 2 * (W.T @ activations)                 # gradient of the loss w.r.t. the image
 
lr = 0.05
img = image.copy()
loss_before = compute_loss(img)
 
for step in range(10):
    grad = compute_grad(img)
    grad = grad / (np.sqrt(np.mean(grad ** 2)) + 1e-8)   # L2-normalize, like the book
    img = img + lr * grad                                 # ASCENT: add, don't subtract
 
loss_after = compute_loss(img)
print("loss before:", round(loss_before, 4))
print("loss after 10 ascent steps:", round(loss_after, 4))
print("loss increased:", loss_after > loss_before)
text
loss before: 0.2347
loss after 10 ascent steps: 4.5998
loss increased: True
text
loss before: 0.2347
loss after 10 ascent steps: 4.5998
loss increased: True

Ten small steps nearly 20x’d the activation — that’s the entire DeepDream mechanism, minus the convnet and the pretty pixels.

Octaves: processing the image at multiple scales

A single pass of gradient ascent at one resolution tends to produce fairly uniform, small-scale texture. DeepDream instead processes the image at several octaves — progressively larger scales — running a handful of ascent steps at each one, then upscaling and re-injecting the detail that upscaling lost. Smaller scales tend to shape broad composition; larger scales add fine texture on top of it:

Building the octave scale sequence
original_shape = (400, 300)   # (height, width) of the real image
num_octave = 3
octave_scale = 1.4
 
successive_shapes = [original_shape]
for i in range(1, num_octave):
    shape = tuple(int(dim / (octave_scale ** i)) for dim in original_shape)
    successive_shapes.append(shape)
successive_shapes = successive_shapes[::-1]   # smallest first, so we scale UP
 
print("octave shapes (smallest to largest):", successive_shapes)
Building the octave scale sequence
original_shape = (400, 300)   # (height, width) of the real image
num_octave = 3
octave_scale = 1.4
 
successive_shapes = [original_shape]
for i in range(1, num_octave):
    shape = tuple(int(dim / (octave_scale ** i)) for dim in original_shape)
    successive_shapes.append(shape)
successive_shapes = successive_shapes[::-1]   # smallest first, so we scale UP
 
print("octave shapes (smallest to largest):", successive_shapes)
text
octave shapes (smallest to largest): [(204, 153), (285, 214), (400, 300)]
text
octave shapes (smallest to largest): [(204, 153), (285, 214), (400, 300)]

The outer loop runs gradient ascent at (204, 153)(204, 153) first, upscales to (285, 214)(285, 214) and runs more ascent steps, then upscales again to the full (400, 300)(400, 300) — each upscale re-injects the fine detail that a naive resize would have blurred away, so the final image stays sharp instead of pixelated.

Visualize it

Watch a small noisy pattern get amplified step by step, exactly like repeated gradient-ascent steps on a real DeepDream image — with each “octave” boundary (dashed line) marking an upscale, after which amplification continues at a larger effective scale:

sketch Iterative pattern amplification across octaves p5.js
A small pattern is repeatedly nudged (gradient ascent) to amplify its own strongest features. Dashed vertical lines mark octave boundaries, where the pattern is upscaled and amplification continues.

Mini-checkpoint

Why does DeepDream start from a real photograph instead of random noise, unlike the filter-visualization technique it’s built on?

  • Starting from a real image means the gradient-ascent process has existing edges, textures, and shapes to latch onto and amplify — the resulting dream distorts and exaggerates elements that were already faintly there, which is what gives DeepDream its “hallucinating on top of reality” look, rather than generating an abstract pattern from scratch.

🧪 Try It Yourself

Exercise 1 – Compute the DeepDream-Style Loss

Exercise 2 – Take a Gradient Ascent Step

Exercise 3 – Build the Octave Shape Sequence

Next

Continue to Neural Style Transfer — instead of amplifying one image’s own features, blend the content of one image with the style of another by optimizing a combined loss.

If this helped you, consider buying me a coffee ☕

Buy me a coffee

Was this page helpful?

Let us know how we did