First Example: Classifying Movie Reviews (IMDB, Binary)

Time to put every piece from this phase together — tensors, layers, activations, gradient descent — into one small, complete project. You’ll classify 50,000movie reviews from IMDB as positive or negative: the “hello world” of binary classification.

The IMDB dataset

The IMDB dataset ships pre-packaged with Keras: 25,000 training and 25,000 test reviews, perfectly balanced between positive and negative. Each review has already been turned into a list of integers, one per word, using a fixed 10,000-word dictionary:

Loading the IMDB dataset

from tensorflow.keras.datasets import imdb
 
(train_data, train_labels), (test_data, test_labels) = imdb.load_data(num_words=10000)
 
print(train_data[0][:10])   # a review, as word indices: [1, 14, 22, 16, ...]
print(train_labels[0])      # 1 -- positive
print(max(max(seq) for seq in train_data))   # 9999 -- capped by num_words

Loading the IMDB dataset

from tensorflow.keras.datasets import imdb
 
(train_data, train_labels), (test_data, test_labels) = imdb.load_data(num_words=10000)
 
print(train_data[0][:10])   # a review, as word indices: [1, 14, 22, 16, ...]
print(train_labels[0])      # 1 -- positive
print(max(max(seq) for seq in train_data))   # 9999 -- capped by num_words

num_words=10000num_words=10000 keeps only the 10,000 most frequent words; rarer words (which mostly appear once and don’t help classification) are dropped so the vocabulary stays a manageable size.

Preparing the data: multi-hot encoding

A neural network needs same-shaped tensors, but reviews have different lengths. The simplest fix: multi-hot encode every review into a fixed-size vector of 0s and 1s — one slot per word in the dictionary, set to 1 if that word appears anywhere in the review.

Multi-hot encoding reviews into fixed-length vectors

import numpy as np
 
def vectorize_sequences(sequences, dimension=10000):
    results = np.zeros((len(sequences), dimension))
    for i, sequence in enumerate(sequences):
        for j in sequence:
            results[i, j] = 1.
    return results
 
x_train = vectorize_sequences(train_data)
x_test = vectorize_sequences(test_data)
 
y_train = np.asarray(train_labels).astype("float32")
y_test = np.asarray(test_labels).astype("float32")
 
print(x_train[0][:20])   # mostly 0s, with 1s where a word index occurred
print(x_train.shape)     # (25000, 10000)

Multi-hot encoding reviews into fixed-length vectors

import numpy as np
 
def vectorize_sequences(sequences, dimension=10000):
    results = np.zeros((len(sequences), dimension))
    for i, sequence in enumerate(sequences):
        for j in sequence:
            results[i, j] = 1.
    return results
 
x_train = vectorize_sequences(train_data)
x_test = vectorize_sequences(test_data)
 
y_train = np.asarray(train_labels).astype("float32")
y_test = np.asarray(test_labels).astype("float32")
 
print(x_train[0][:20])   # mostly 0s, with 1s where a word index occurred
print(x_train.shape)     # (25000, 10000)

Building the model

The input is a vector, the label is a single 0/1 scalar — one of the simplest setups you’ll meet. A plain stack of DenseDense layers with relurelu does the job well:

A 3-layer Dense stack for binary classification

from tensorflow import keras
from tensorflow.keras import layers
 
model = keras.Sequential([
    layers.Dense(16, activation="relu"),
    layers.Dense(16, activation="relu"),
    layers.Dense(1, activation="sigmoid"),
])
 
model.compile(optimizer="rmsprop",
              loss="binary_crossentropy",
              metrics=["accuracy"])

A 3-layer Dense stack for binary classification

from tensorflow import keras
from tensorflow.keras import layers
 
model = keras.Sequential([
    layers.Dense(16, activation="relu"),
    layers.Dense(16, activation="relu"),
    layers.Dense(1, activation="sigmoid"),
])
 
model.compile(optimizer="rmsprop",
              loss="binary_crossentropy",
              metrics=["accuracy"])

Two 16-unit hidden layers with relurelu — 16 is the dimensionality of the representation space each layer is allowed to learn. More units means more capacity, but also more risk of overfitting and more compute.
The output layer has one unit and a sigmoid activation, so the model outputs a single probability between 0 and 1 (how likely the review is positive).
binary_crossentropybinary_crossentropy is the standard loss when a model outputs a probability for a two-class problem — it measures the distance between two probability distributions (predicted vs. true).
rmsproprmsprop is a solid, low-effort default optimizer for almost any problem.

diagram The IMDB classification pipeline mermaid

  flowchart LR
  A["Raw review
(word indices)"] --> B["Multi-hot vector
(10000,)"]
  B --> C["Dense(16, relu)"] --> D["Dense(16, relu)"] --> E["Dense(1, sigmoid)"]
  E --> F["P(positive)"]

Raw reviews are vectorized, fed through two ReLU hidden layers, and a sigmoid layer outputs a positive/negative probability.

Validating and training

Never evaluate on training data alone — hold out a validation set to watch for overfitting during training:

Training with a held-out validation set

x_val = x_train[:10000]
partial_x_train = x_train[10000:]
y_val = y_train[:10000]
partial_y_train = y_train[10000:]
 
history = model.fit(partial_x_train, partial_y_train,
                     epochs=20, batch_size=512,
                     validation_data=(x_val, y_val))
 
history_dict = history.history
print(history_dict.keys())
# dict_keys(['loss', 'accuracy', 'val_loss', 'val_accuracy'])

Training with a held-out validation set

x_val = x_train[:10000]
partial_x_train = x_train[10000:]
y_val = y_train[:10000]
partial_y_train = y_train[10000:]
 
history = model.fit(partial_x_train, partial_y_train,
                     epochs=20, batch_size=512,
                     validation_data=(x_val, y_val))
 
history_dict = history.history
print(history_dict.keys())
# dict_keys(['loss', 'accuracy', 'val_loss', 'val_accuracy'])

Plotting history_dict["loss"]history_dict["loss"] against history_dict["val_loss"]history_dict["val_loss"] typically shows training loss falling every epoch — but validation loss bottoming out around epoch 4 and then climbing. That’s overfitting: the model starts memorizing quirks of the training set that don’t generalize. The fix here is simple — stop training once validation performance peaks:

Retraining for the right number of epochs

model = keras.Sequential([
    layers.Dense(16, activation="relu"),
    layers.Dense(16, activation="relu"),
    layers.Dense(1, activation="sigmoid"),
])
model.compile(optimizer="rmsprop", loss="binary_crossentropy", metrics=["accuracy"])
model.fit(x_train, y_train, epochs=4, batch_size=512)
 
results = model.evaluate(x_test, y_test)
print(results)   # roughly [0.29, 0.88] -- test loss, test accuracy

Retraining for the right number of epochs

model = keras.Sequential([
    layers.Dense(16, activation="relu"),
    layers.Dense(16, activation="relu"),
    layers.Dense(1, activation="sigmoid"),
])
model.compile(optimizer="rmsprop", loss="binary_crossentropy", metrics=["accuracy"])
model.fit(x_train, y_train, epochs=4, batch_size=512)
 
results = model.evaluate(x_test, y_test)
print(results)   # roughly [0.29, 0.88] -- test loss, test accuracy

This simple approach reaches about 88% test accuracy — not bad for a first try with plain DenseDense layers and no fine-tuning.

Predicting on new data

Generating predictions

predictions = model.predict(x_test)
print(predictions[:3])
# array([[0.98], [0.03], [0.65]])  -- confident positive, confident negative, unsure

Generating predictions

predictions = model.predict(x_test)
print(predictions[:3])
# array([[0.98], [0.03], [0.65]])  -- confident positive, confident negative, unsure

Values near 0 or 1 mean the model is confident; values near 0.5 mean it’s unsure.

Mini-checkpoint

Why does the output layer use sigmoidsigmoid instead of no activation at all?

sigmoidsigmoid squashes any real number into (0, 1)(0, 1), so the output can be read as a probability — which is exactly what binary_crossentropybinary_crossentropy expects.

Binary classification was two classes. First Example: Classifying Newswires (Reuters, Multiclass) tackles the same kind of problem with 46 classes instead of 2 — and shows what changes (and what doesn’t) when the number of outputs grows.

First Example: Classifying Movie Reviews (IMDB, Binary)

The IMDB dataset

Preparing the data: multi-hot encoding

Building the model

Validating and training

Predicting on new data

Mini-checkpoint

Next

🧪 Try It Yourself

Exercise 1 – Multi-Hot Encode a Review

Exercise 2 – Build the Binary Classifier

Exercise 3 – Compile with the Right Loss

Was this page helpful?