What you’ll learn

• how TP, TN, FP, and FN are defined, and how to read them off a 2x2 grid
• how to compute a confusion matrix by hand from a small worked example
• how to read a real, book-scale confusion matrix (rows vs columns)
• how confusion matrices extend to more than two classes, and how to spot which classes get confused with each other
• common gotchas: row/column orientation, class order, and normalizing counts

What the confusion matrix shows

A confusion matrix compares:

• actual labels
• predicted labels

For binary classification:

• TP (true positive): predicted positive, actually positive
• TN (true negative): predicted negative, actually negative
• FP (false positive): predicted positive, actually negative — a “false alarm”
• FN (false negative): predicted negative, actually positive — a “miss”

  flowchart TD
  A[Actual Positive] -->|Pred Positive| TP[TP]
  A -->|Pred Negative| FN[FN]
  B[Actual Negative] -->|Pred Negative| TN[TN]
  B -->|Pred Positive| FP[FP]

Why it matters

All important classification metrics — precision, recall, F1, specificity, even ROC/AUC — are computed from nothing but these four counts. The confusion matrix is the raw material; every other metric on this phase’s remaining pages is just a different way of summarizing it.

A worked example, cell by cell

Say a spam filter checks 20 emails. 8 are actually spam, 12 are actually not spam (“ham”). The filter flags 9 emails as spam:

• of the 8 real spam emails, it correctly flags 6 (TP = 6) and misses 2 (FN = 2)
• of the 12 real ham emails, it wrongly flags 3 as spam (FP = 3) and correctly leaves 9 alone (TN = 9)

import numpy as np
 
TP, FN, FP, TN = 6, 2, 3, 9
 
cm = np.array([
    [TN, FP],   # actual ham:  correctly ham,  wrongly flagged spam
    [FN, TP],   # actual spam: wrongly missed,  correctly flagged
])
print(cm)
# [[9 3]
#  [2 6]]
 
total = cm.sum()
accuracy = (TP + TN) / total
print("Accuracy:", round(accuracy, 3))
# Accuracy: 0.75

import numpy as np
 
TP, FN, FP, TN = 6, 2, 3, 9
 
cm = np.array([
    [TN, FP],   # actual ham:  correctly ham,  wrongly flagged spam
    [FN, TP],   # actual spam: wrongly missed,  correctly flagged
])
print(cm)
# [[9 3]
#  [2 6]]
 
total = cm.sum()
accuracy = (TP + TN) / total
print("Accuracy:", round(accuracy, 3))
# Accuracy: 0.75

Notice the matrix layout: rows are the actual class, columns are the predicted class — this is scikit-learn’s convention (and the one this page uses throughout). Reading along the diagonal gives every correct prediction; everything off the diagonal is a mistake the model made.

Reading a real confusion matrix

Géron’s book computes the confusion matrix for the MNIST “5-detector” from the Introduction to Classification page, using cross_val_predict()cross_val_predict() so every prediction comes from a model that never saw that instance during training:

from sklearn.metrics import confusion_matrix
 
# y_train_5: True for real 5s, y_train_pred: model's cross-validated predictions
cm = confusion_matrix(y_train_5, y_train_pred)
print(cm)
# [[53057  1522]
#  [ 1325  4096]]

from sklearn.metrics import confusion_matrix
 
# y_train_5: True for real 5s, y_train_pred: model's cross-validated predictions
cm = confusion_matrix(y_train_5, y_train_pred)
print(cm)
# [[53057  1522]
#  [ 1325  4096]]

Rows are the actual class, columns are the predicted class:

• row 0 (actual “not 5”): 53,057 correctly called not-5 (TN), 1,522 wrongly called 5 (FP)
• row 1 (actual “5”): 1,325 wrongly called not-5 (FN), 4,096 correctly caught (TP)

A perfect classifier would only ever have nonzero values on the main diagonal — every off-diagonal cell is a mistake.

Visualize it

Each prediction lands in exactly one of the four cells. Watch the stream of incoming predictions on the left fly into the matching cell on the right, and the counts build up live — this is exactly what confusion_matrix()confusion_matrix() does internally, just one instance at a time instead of all at once:

Multiclass confusion matrices

With more than two classes, the confusion matrix simply grows to N x NN x N — row ii, column jj counts how many instances of actual class ii got predicted as class jj. The book builds one for all 10 MNIST digits:

from sklearn.metrics import confusion_matrix
 
# y_train_pred here comes from cross_val_predict() on the full 10-class problem
conf_mx = confusion_matrix(y_train, y_train_pred)
print(conf_mx.shape)
# (10, 10)

from sklearn.metrics import confusion_matrix
 
# y_train_pred here comes from cross_val_predict() on the full 10-class problem
conf_mx = confusion_matrix(y_train, y_train_pred)
print(conf_mx.shape)
# (10, 10)

With 100 numbers to scan, raw counts are hard to read at a glance — Géron plots it as an image instead (plt.matshow(conf_mx, cmap=plt.cm.gray)plt.matshow(conf_mx, cmap=plt.cm.gray)), where brighter cells mean bigger counts. Most images land on the main diagonal (correct predictions), so a well-behaved classifier’s matshow plot looks like a bright diagonal line on a dark background.

To spot errors specifically, normalize each row by its class size (so common classes don’t automatically look “worse” just for having more examples), then zero out the diagonal to hide the correct predictions entirely:

import numpy as np
 
row_sums = conf_mx.sum(axis=1, keepdims=True)
norm_conf_mx = conf_mx / row_sums
np.fill_diagonal(norm_conf_mx, 0)
# now the brightest remaining cells reveal exactly which classes
# get confused with which -- e.g. Géron finds 3s and 5s often swap

import numpy as np
 
row_sums = conf_mx.sum(axis=1, keepdims=True)
norm_conf_mx = conf_mx / row_sums
np.fill_diagonal(norm_conf_mx, 0)
# now the brightest remaining cells reveal exactly which classes
# get confused with which -- e.g. Géron finds 3s and 5s often swap

This is also exactly what scikit-learn’s normalizenormalize argument does for you directly: confusion_matrix(y_true, y_pred, normalize="true")confusion_matrix(y_true, y_pred, normalize="true") divides each row by its total in one call.

Common pitfalls

• Orientation: scikit-learn always puts actual classes on rows, predicted classes on columns — but some textbooks and plotting libraries transpose this. Always check which axis is which before reading a matrix you didn’t build yourself.
• Class order: confusion_matrix()confusion_matrix() sorts labels automatically unless you pass labels=[...]labels=[...] explicitly — don’t assume row/column 0 means “class 0” on datasets with non-numeric or non-sorted labels.
• Raw counts vs. rates: on imbalanced data, raw counts make the majority class dominate the picture. Normalizing by row (as above) is usually the more honest comparison.
• A confusion matrix by itself isn’t a single score you can rank models by — that’s exactly what precision, recall, F1, and AUC (the next pages) are for.

Scikit-learn example

from sklearn.metrics import confusion_matrix
 
cm = confusion_matrix(y_true, y_pred)
print(cm)

from sklearn.metrics import confusion_matrix
 
cm = confusion_matrix(y_true, y_pred)
print(cm)

Mini-checkpoint

If FN is very high, what does that mean?

• You’re missing many real positives.

🧪 Try It Yourself

Exercise 1 – Extract TP, TN, FP, FN from a Matrix

Exercise 2 – Compute Accuracy From the Matrix

Exercise 3 – Build and Read a Confusion Matrix

Next

Continue to Precision, Recall, and F1-Score to turn these four numbers into the metrics you’ll actually report and compare.