What you’ll learn

• why a small dataset makes overfitting your #1 enemy, not underfitting
• how RandomFlipRandomFlip, RandomRotationRandomRotation, and RandomZoomRandomZoom generate new believable training images on the fly
• how to chain augmentation layers into a SequentialSequential “augmentation stage” and bolt it onto the front of a CNN
• why augmentation layers (like DropoutDropout) are only active during training, never during evaluation or prediction

Why small datasets overfit

Deep learning models need to see many examples of the patterns they’re supposed to learn. If you only have a few thousand images — say, 2,000 pictures of cats and dogs — a convnet trained from scratch will quickly memorize the training set instead of learning generalizable rules. Training accuracy climbs toward 100% while validation accuracy stalls, then falls behind: classic overfitting.

You already know two general-purpose fixes: DropoutDropout and weight decay (L2 regularization). Computer vision has one more trick that’s used almost everywhere: data augmentation.

Data augmentation: manufacturing believable variety

Given infinite training data, a model would see every possible variation of the input distribution and simply couldn’t overfit. Data augmentation gets you a little closer to that ideal by generating more training images from the ones you already have — applying random, realistic transformations (flips, small rotations, zooms) so that the model never sees the exact same picture twice. A photo of a cat, flipped horizontally and rotated a few degrees, is still unmistakably a photo of a cat — but it’s pixel-for-pixel different from the original, which is exactly what the model needs to generalize better.

In Keras, this is just a stack of preprocessing layers you place at the very start of your model:

from tensorflow import keras
from tensorflow.keras import layers
 
data_augmentation = keras.Sequential([
    layers.RandomFlip("horizontal"),
    layers.RandomRotation(0.1),
    layers.RandomZoom(0.2),
])

from tensorflow import keras
from tensorflow.keras import layers
 
data_augmentation = keras.Sequential([
    layers.RandomFlip("horizontal"),
    layers.RandomRotation(0.1),
    layers.RandomZoom(0.2),
])

• RandomFlip("horizontal")RandomFlip("horizontal") — mirrors a random 50% of the images left-to-right.
• RandomRotation(0.1)RandomRotation(0.1) — rotates by a random fraction of a full circle, here ±10% (≈ ±36°).
• RandomZoom(0.2)RandomZoom(0.2) — zooms in or out by a random factor, here ±20%.

  flowchart LR
  I["Training image"] --> F["RandomFlip
(horizontal)"]
  F --> R["RandomRotation
(+/-10%)"]
  R --> Z["RandomZoom
(+/-20%)"]
  Z --> C["Rest of the CNN
(Rescaling -> Conv2D -> ...)"]

Wiring it into a full model

The augmentation stage is just another layer, so you place it right after the InputInput and before RescalingRescaling. Because it can’t manufacture genuinely new information — only remix what’s already in your training images — it’s usually paired with DropoutDropout right before the final classifier for a stronger combined effect:

from tensorflow import keras
from tensorflow.keras import layers
 
data_augmentation = keras.Sequential([
    layers.RandomFlip("horizontal"),
    layers.RandomRotation(0.1),
    layers.RandomZoom(0.2),
])
 
inputs = keras.Input(shape=(180, 180, 3))
x = data_augmentation(inputs)
x = layers.Rescaling(1. / 255)(x)
x = layers.Conv2D(filters=32, kernel_size=3, activation="relu")(x)
x = layers.MaxPooling2D(pool_size=2)(x)
x = layers.Conv2D(filters=64, kernel_size=3, activation="relu")(x)
x = layers.MaxPooling2D(pool_size=2)(x)
x = layers.Conv2D(filters=128, kernel_size=3, activation="relu")(x)
x = layers.MaxPooling2D(pool_size=2)(x)
x = layers.Conv2D(filters=256, kernel_size=3, activation="relu")(x)
x = layers.Flatten()(x)
x = layers.Dropout(0.5)(x)
outputs = layers.Dense(1, activation="sigmoid")(x)
model = keras.Model(inputs=inputs, outputs=outputs)
 
model.compile(loss="binary_crossentropy", optimizer="rmsprop", metrics=["accuracy"])

from tensorflow import keras
from tensorflow.keras import layers
 
data_augmentation = keras.Sequential([
    layers.RandomFlip("horizontal"),
    layers.RandomRotation(0.1),
    layers.RandomZoom(0.2),
])
 
inputs = keras.Input(shape=(180, 180, 3))
x = data_augmentation(inputs)
x = layers.Rescaling(1. / 255)(x)
x = layers.Conv2D(filters=32, kernel_size=3, activation="relu")(x)
x = layers.MaxPooling2D(pool_size=2)(x)
x = layers.Conv2D(filters=64, kernel_size=3, activation="relu")(x)
x = layers.MaxPooling2D(pool_size=2)(x)
x = layers.Conv2D(filters=128, kernel_size=3, activation="relu")(x)
x = layers.MaxPooling2D(pool_size=2)(x)
x = layers.Conv2D(filters=256, kernel_size=3, activation="relu")(x)
x = layers.Flatten()(x)
x = layers.Dropout(0.5)(x)
outputs = layers.Dense(1, activation="sigmoid")(x)
model = keras.Model(inputs=inputs, outputs=outputs)
 
model.compile(loss="binary_crossentropy", optimizer="rmsprop", metrics=["accuracy"])

On a 2,000-image cats-vs-dogs dataset, a plain convnet like this one starts overfitting almost immediately — around epoch 10 — and tops out near 70% test accuracy. Add the augmentation stage and dropout, train for three times as many epochs (say, 100, since overfitting now takes much longer to kick in), and overfitting doesn’t show up until around epoch 60–70 — pushing validation accuracy up into the 80–85% range.

Visualize it

Every time the same source image passes through the augmentation stage, it comes out a little different — flipped, rotated a few degrees, zoomed in or out. The six variants below never stop drifting through new combinations, so you can watch the augmentation stage continuously resample believable new versions of the same picture. Click the canvas to jump to a fresh batch:

Mini-checkpoint

Why does adding a DropoutDropout layer alongside data augmentation help more than either technique on its own?

• Augmentation can only remix the information already present in your small set of source images — it can’t invent genuinely new information. Dropout attacks overfitting from a different angle, forcing the dense classifier to not rely on any single feature too heavily. Together they push back overfitting from two directions at once.

🧪 Try It Yourself

Exercise 1 – Build a Data Augmentation Stage

Exercise 2 – Augmenting Keeps the Image Shape

Exercise 3 – Inactive at Inference

Next

Continue to Image Segmentation — classifying every single pixel of an image, not just the image as a whole, using an encoder-decoder convnet.