What you’ll learn

• why sentiment analysis is often called the “hello world” of NLP
• a fast, strong baseline: TF-IDF + Logistic Regression
• how to go deeper with a real sequence model: Embedding + RNN, learned end-to-end
• why masking matters once your reviews are padded to the same length
• how to reuse someone else’s pretrained embeddings instead of learning your own
• how to read a model’s raw output as a sentiment score

Goal

Classify text as positive or negative sentiment.

If MNIST is the “hello world” of computer vision, the IMDb movie reviews dataset plays that role for NLP: 50,000 reviews, half positive and half negative, simple enough to tackle on a laptop, but rewarding enough to actually learn from. It’s a perfect first NLP project because:

• preprocessing matters — how you turn text into numbers shapes everything downstream
• classic models work surprisingly well as a baseline before you reach for deep learning

A baseline approach: TF-IDF + Logistic Regression

  flowchart LR
  T["Raw text"] --> V["TF-IDF
(sparse features)"]
  V --> C["Classifier"]
  C --> S["Sentiment score
0 = negative, 1 = positive"]

from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import classification_report
 
texts = [
    "I love this product",
    "This is terrible",
    "Amazing quality",
    "Worst purchase ever",
]
labels = [1, 0, 1, 0]
 
X_train, X_test, y_train, y_test = train_test_split(
    texts, labels, test_size=0.25, random_state=42, stratify=labels
)
 
model = Pipeline(
    steps=[
        ("tfidf", TfidfVectorizer(ngram_range=(1, 2))),
        ("clf", LogisticRegression(max_iter=1000)),
    ]
)
 
model.fit(X_train, y_train)
 
y_pred = model.predict(X_test)
print(classification_report(y_test, y_pred))

from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import classification_report
 
texts = [
    "I love this product",
    "This is terrible",
    "Amazing quality",
    "Worst purchase ever",
]
labels = [1, 0, 1, 0]
 
X_train, X_test, y_train, y_test = train_test_split(
    texts, labels, test_size=0.25, random_state=42, stratify=labels
)
 
model = Pipeline(
    steps=[
        ("tfidf", TfidfVectorizer(ngram_range=(1, 2))),
        ("clf", LogisticRegression(max_iter=1000)),
    ]
)
 
model.fit(X_train, y_train)
 
y_pred = model.predict(X_test)
print(classification_report(y_test, y_pred))

This baseline is fast, needs no GPU, and is genuinely hard to beat on small datasets. But it treats every word (or bigram) as an independent feature — it has no idea “great” and “fantastic” mean almost the same thing, and it can’t use word order beyond two-word windows.

Going deeper: a sequence model that reads word by word

A recurrent model reads the review one word at a time, building up a hidden state as it goes, and only makes its prediction once it’s seen the whole sequence:

import tensorflow as tf
 
vocab_size = 10_000       # top N most frequent words, plus out-of-vocabulary buckets
embed_size = 128
 
model = tf.keras.Sequential([
    # mask_zero=True tells every downstream layer to ignore <pad> tokens (ID 0)
    tf.keras.layers.Embedding(vocab_size, embed_size, mask_zero=True, input_shape=[None]),
    tf.keras.layers.GRU(128, return_sequences=True),
    tf.keras.layers.GRU(128),
    tf.keras.layers.Dense(1, activation="sigmoid"),
])
 
model.compile(loss="binary_crossentropy", optimizer="adam", metrics=["accuracy"])
model.summary()

import tensorflow as tf
 
vocab_size = 10_000       # top N most frequent words, plus out-of-vocabulary buckets
embed_size = 128
 
model = tf.keras.Sequential([
    # mask_zero=True tells every downstream layer to ignore <pad> tokens (ID 0)
    tf.keras.layers.Embedding(vocab_size, embed_size, mask_zero=True, input_shape=[None]),
    tf.keras.layers.GRU(128, return_sequences=True),
    tf.keras.layers.GRU(128),
    tf.keras.layers.Dense(1, activation="sigmoid"),
])
 
model.compile(loss="binary_crossentropy", optimizer="adam", metrics=["accuracy"])
model.summary()

The EmbeddingEmbedding layer turns each word ID into a dense vector, so the model input goes from [batch, time steps][batch, time steps] to [batch, time steps, embed_size][batch, time steps, embed_size]. The first GRUGRU layer keeps return_sequences=Truereturn_sequences=True so it can pass a full sequence of hidden states to the second GRUGRU; the second GRUGRU only returns its last hidden state, since sentiment is a many-to-one problem — one review, one label. The final Dense(1, "sigmoid")Dense(1, "sigmoid") squashes that into a single probability: how positive the review is.

Because reviews in a batch have different lengths, shorter ones get padded with token ID 00. mask_zero=Truemask_zero=True on the EmbeddingEmbedding layer tells every downstream layer to simply skip those padded positions — so the model never wastes capacity “learning” that padding means nothing.

Reusing pretrained embeddings

Training good word embeddings from scratch takes a lot of text — 25,000 reviews is a small corpus. TensorFlow Hub makes it easy to plug in embeddings pretrained on a much bigger corpus instead:

import tensorflow as tf
import tensorflow_hub as hub
 
model = tf.keras.Sequential([
    hub.KerasLayer(
        "https://tfhub.dev/google/tf2-preview/nnlm-en-dim50/1",
        dtype=tf.string, input_shape=[], output_shape=[50],
    ),
    tf.keras.layers.Dense(128, activation="relu"),
    tf.keras.layers.Dense(1, activation="sigmoid"),
])
 
model.compile(loss="binary_crossentropy", optimizer="adam", metrics=["accuracy"])

import tensorflow as tf
import tensorflow_hub as hub
 
model = tf.keras.Sequential([
    hub.KerasLayer(
        "https://tfhub.dev/google/tf2-preview/nnlm-en-dim50/1",
        dtype=tf.string, input_shape=[], output_shape=[50],
    ),
    tf.keras.layers.Dense(128, activation="relu"),
    tf.keras.layers.Dense(1, activation="sigmoid"),
])
 
model.compile(loss="binary_crossentropy", optimizer="adam", metrics=["accuracy"])

This hub.KerasLayerhub.KerasLayer takes raw strings directly — no manual tokenizing or padding needed. Internally it splits on spaces, embeds each word using weights trained on the 7-billion-word Google News corpus, and averages them into a single 50-dimensional sentence vector. By default the layer is frozen; pass trainable=Truetrainable=True when creating it to fine-tune those embeddings for your own task.

Do you even need the RNN? A rule of thumb for IMDb

Before reaching for an Embedding + RNN model, it’s worth asking whether the TF-IDF baseline is already good enough. Chollet’s rule of thumb: look at the ratio of training samples ÷ average words per sample. Below roughly 1,500, a bag-of-bigrams model tends to win; above it, a sequence model tends to win.

For the IMDb dataset specifically — 20,000 training reviews, averaging about 233 words each — that ratio is only about 86. The heuristic says bag-of-words should win here, and in Chollet’s own experiments it does: a plain bigram + TF-IDF model reached roughly 89-90% test accuracy, matching or slightly beating an Embedding + bidirectional LSTM sequence model on the same data. Sequence models only pull ahead once you have a lot more (or much shorter) training examples to learn from.

from tensorflow.keras.layers import TextVectorization
from tensorflow import keras
 
text_vectorization = TextVectorization(
    ngrams=2,             # reinject a bit of word-order information
    max_tokens=20000,
    output_mode="tf_idf", # term-frequency / inverse-document-frequency weighting
)
# text_only_train_ds yields just the raw review strings (no labels)
text_vectorization.adapt(text_only_train_ds)
 
# a couple of Dense layers on top of these TF-IDF vectors is often
# every bit as accurate as a much heavier RNN, and far cheaper to train
model = keras.Sequential([
    keras.layers.Dense(16, activation="relu"),
    keras.layers.Dropout(0.5),
    keras.layers.Dense(1, activation="sigmoid"),
])

from tensorflow.keras.layers import TextVectorization
from tensorflow import keras
 
text_vectorization = TextVectorization(
    ngrams=2,             # reinject a bit of word-order information
    max_tokens=20000,
    output_mode="tf_idf", # term-frequency / inverse-document-frequency weighting
)
# text_only_train_ds yields just the raw review strings (no labels)
text_vectorization.adapt(text_only_train_ds)
 
# a couple of Dense layers on top of these TF-IDF vectors is often
# every bit as accurate as a much heavier RNN, and far cheaper to train
model = keras.Sequential([
    keras.layers.Dense(16, activation="relu"),
    keras.layers.Dropout(0.5),
    keras.layers.Dense(1, activation="sigmoid"),
])

Practical tips

• include bigrams (ngram_range=(1, 2)ngram_range=(1, 2)) with TF-IDF to capture phrases like “not good”
• never strip out negation words like “not” during preprocessing — they flip the whole meaning
• check class balance before trusting accuracy alone; use classification_reportclassification_report for precision/recall per class
• you often don’t need the whole review — the first sentence or two usually gives away the sentiment

Visualize it

A sentiment model’s raw output is a single number between 0 and 1. It helps to picture that number as a gauge needle sweeping from very negative to very positive:

Mini-checkpoint

Why is “not good” tricky for a plain bag-of-words model?

• Without bigrams, “not” and “good” are counted as two separate, unrelated features — the model never sees them as one phrase, so their individual signals (negative-ish “not”, positive “good”) can cancel out incorrectly instead of combining into a clearly negative signal.

🧪 Try It Yourself

Exercise 1 – Fit a TF-IDF + Logistic Regression Classifier

Exercise 2 – An Embedding Layer’s Output Shape

Exercise 3 – Averaging Word Vectors Into a Sentence Vector

Next

Continue to Named Entity Recognition (NER) — instead of one label per review, learn to label every single token in a sentence.