Why CNNs exist

Images have structure:

• nearby pixels are related
• patterns repeat across the image

A regular DenseDense layer ignores both facts — it flattens the image into a long 1D list and connects every pixel to every neuron. For a small 28 × 28 MNIST digit that’s already wasteful; for a 100 × 100 photo, a first hidden layer of just 1,000 neurons would need 10 million connections. CNNs solve this using partially connected layers and weight sharing: instead of looking at the whole image at once, a neuron only looks at a small patch of it, and that same small filter is reused everywhere across the image.

Where the idea came from

CNNs trace back to biology. In 1958–59, Hubel and Wiesel discovered that neurons in the visual cortex have a small local receptive field — they only react to a limited region of the visual field — and that simple detectors (like a neuron that only fires for horizontal lines) feed into higher-level neurons that combine them into more complex patterns. That hierarchical structure — simple filters low down, complex combinations higher up — is exactly what a stack of convolutional layers reproduces.

Convolutional layers: local receptive fields

In a convolutional layer, a neuron at row ii, column jj is connected only to a small rectangle of the previous layer (its receptive field), not to every pixel. Two hyperparameters control how that rectangle moves:

• stride — how far the receptive field shifts each step. A stride of 2 skips every other position, roughly halving the output’s height and width.
• padding — "valid""valid" uses no padding (the output shrinks a little each layer); "same""same" adds zeros around the border so the output keeps the same height and width as the input when the stride is 1.

Filters and feature maps

A neuron’s weights can be drawn as a small image the size of its receptive field — this is called a filter (or convolution kernel). For example, a filter that is all zeros except for a vertical line of 11s in the middle will ignore everything in its receptive field except the vertical line passing through it. If every neuron in a layer uses that same filter, the whole layer produces a feature map: an image that lights up wherever the input matches the filter, and stays dark everywhere else.

You don’t have to hand-design these filters. During training, the convolutional layer learns whichever filters are most useful for the task — edges and color blobs in the early layers, then textures, then whole shapes higher up the stack.

Stacking multiple feature maps

A real convolutional layer applies many filters at once, so it outputs one feature map per filter — making its output a 3D volume rather than a flat image. All the neurons within one feature map share the same weights and bias (that’s the weight sharing), but neurons in different feature maps learn different filters. Once a CNN learns to recognize a pattern in one spot, it can recognize that same pattern anywhere else in the image — a regular DenseDense network has no such luck.

  flowchart LR
  I["Image
(H, W, channels)"] --> K1["Filter 1"]
  I --> K2["Filter 2"]
  I --> K3["Filter 3"]
  K1 --> F1["Feature map 1"]
  K2 --> F2["Feature map 2"]
  K3 --> F3["Feature map 3"]
  F1 --> S["Stack into one 3D output
(H', W', filters)"]
  F2 --> S
  F3 --> S

tf.keras implementation

You can see convolution happen directly with tf.nn.conv2dtf.nn.conv2d, by hand-crafting a filter that only reacts to vertical edges:

import numpy as np
import tensorflow as tf
 
# A tiny 8x8 grayscale "image" with a bright vertical stripe down the middle
image = np.zeros((8, 8), dtype=np.float32)
image[:, 3:5] = 1.0
images = image.reshape(1, 8, 8, 1)  # TF wants [batch, height, width, channels]
 
# A 3x3 filter that reacts to vertical edges (dark -> bright transitions)
vertical_edge_filter = np.array([
    [-1, 0, 1],
    [-1, 0, 1],
    [-1, 0, 1],
], dtype=np.float32).reshape(3, 3, 1, 1)
 
feature_map = tf.nn.conv2d(images, vertical_edge_filter, strides=1, padding="SAME")
print("Feature map shape:", feature_map.shape)
print(np.round(feature_map.numpy()[0, :, :, 0], 1))

import numpy as np
import tensorflow as tf
 
# A tiny 8x8 grayscale "image" with a bright vertical stripe down the middle
image = np.zeros((8, 8), dtype=np.float32)
image[:, 3:5] = 1.0
images = image.reshape(1, 8, 8, 1)  # TF wants [batch, height, width, channels]
 
# A 3x3 filter that reacts to vertical edges (dark -> bright transitions)
vertical_edge_filter = np.array([
    [-1, 0, 1],
    [-1, 0, 1],
    [-1, 0, 1],
], dtype=np.float32).reshape(3, 3, 1, 1)
 
feature_map = tf.nn.conv2d(images, vertical_edge_filter, strides=1, padding="SAME")
print("Feature map shape:", feature_map.shape)
print(np.round(feature_map.numpy()[0, :, :, 0], 1))

In practice you’ll almost always use the Conv2DConv2D layer instead, and let the network learn the filters during training:

import tensorflow as tf
 
conv = tf.keras.layers.Conv2D(
    filters=32,        # number of learned filters (feature maps)
    kernel_size=3,      # each filter is 3x3
    strides=1,
    padding="same",      # keep the same height/width as the input
    activation="relu",
)

import tensorflow as tf
 
conv = tf.keras.layers.Conv2D(
    filters=32,        # number of learned filters (feature maps)
    kernel_size=3,      # each filter is 3x3
    strides=1,
    padding="same",      # keep the same height/width as the input
    activation="relu",
)

Key parts

• Convolution layers: learn filters
• Pooling: reduce spatial size, keep important features
• Fully connected layers: final classification/regression

Why they work well

• parameter sharing (same filter used across image)
• translation invariance (patterns recognized in different locations)

Downsampling: pooling isn’t the only option

The next page covers pooling layers in depth, but it’s worth knowing early on that a convolutional layer can shrink its own output too — you don’t strictly need a separate pooling layer to downsample. Setting strides=2strides=2 on a Conv2DConv2D layer makes it skip every other position as it slides, which roughly halves the height and width of its output, exactly like a 2 × 2 pooling layer would.

import tensorflow as tf
 
# strides=2 halves height and width, same as MaxPooling2D(pool_size=2) would
downsampling_conv = tf.keras.layers.Conv2D(
    filters=64, kernel_size=3, strides=2, padding="same", activation="relu"
)

import tensorflow as tf
 
# strides=2 halves height and width, same as MaxPooling2D(pool_size=2) would
downsampling_conv = tf.keras.layers.Conv2D(
    filters=64, kernel_size=3, strides=2, padding="same", activation="relu"
)

Most classification convnets still prefer plain pooling for downsampling — it’s cheap (no weights to learn) and its translation invariance is a nice property when you only care “is this pattern present somewhere?” But for tasks where the exact location of a pattern matters — like image segmentation, where you need to predict a class for every pixel — a strided convolution downsamples without throwing away location information the way max pooling does. You’ll see this trade-off again on the Image Segmentation page.

Visualize it

Watch a 3 × 3 kernel slide across an 8 × 8 input, one receptive field at a time. At each position it multiplies the pixels underneath it by the kernel’s weights and sums the result into a single feature-map cell — the amber box is the receptive field being read right now, the green box is the output cell being written:

Mini-checkpoint

Why not use a plain MLP for images?

• too many parameters and ignores spatial structure.

🧪 Try It Yourself

Exercise 1 – Create a Conv2D Layer

Exercise 2 – Convolve an Image With a Filter

Exercise 3 – Stack Filters Into a Feature Volume

Next

Continue to Pooling & CNN Architecture — see how pooling layers shrink these feature maps and how convolution + pooling combine into a full CNN.