Two Ways to Read This Book

§1.2 is two pages long and it is the most practically useful part of Chapter 1, because it answers the question every reader actually has: do I have to read this in order?

The honest answer is no, and the book was structured deliberately so that the answer could be no. It also names the price of each route, which is the part usually skipped.

What you’ll learn

The two reading strategies, and the specific failure mode of each.
The four pillars of machine learning and the six mathematical foundations under them.
Which chapters genuinely depend on which — the graph, not a vague ordering.
How to pick a route for the goal you actually have.
Where the exercises fit, and why Part I’s are pen-and-paper while Part II’s are code.

Intuition: two ways up the same building

Bottom-up builds from foundations to applications. Its advantage is that you never rely on something you have not seen. Its cost, in the book’s own words, is that “for a practitioner many of the foundational concepts are not particularly interesting by themselves, and the lack of motivation means that most foundational definitions are quickly forgotten.”

That is a sharper criticism than it looks. It does not say bottom-up is slow — it says the material does not stick, because you learned it without a reason to.

Top-down starts from what you want to do and drills down to what you need. Now you always know why you are learning something. The cost is that “the knowledge is built on potentially shaky foundations, and the readers have to remember a set of words that they do not have any way of understanding.”

Also sharper than it looks: not that you will be confused, but that you will fluently use words you cannot define. Which works, until it does not.

diagram Diagram mermaid

  flowchart TD
  subgraph BU["bottom-up"]
    B1["foundations first"] --> B2["always standing on solid ground"]
    B2 --> B3["cost: unmotivated definitions
are quickly forgotten"]
  end
  subgraph TD2["top-down"]
    T1["problem first"] --> T2["always know why you are here"]
    T2 --> T3["cost: fluent use of words
you cannot define"]
  end
  B3 --> MIX["what most readers actually do:
alternate between them"]
  T3 --> MIX

The book’s closing note on this is the one to take: “Of course there are more than two ways to read this book.” Most readers mix — some foundations, then a model, then back for whatever the derivation used. This module is built to support that; every page names the sections it depends on and the sections that use it.

The math: four pillars on six foundations

The book’s Figure 1.1 draws machine learning as a building. Four pillars hold up the roof, and each pillar stands on foundations from Part I.

diagram Diagram mermaid

  flowchart TD
  ML["MACHINE LEARNING"]

  P1["Regression
Ch 9"]
  P2["Dimensionality Reduction
Ch 10"]
  P3["Density Estimation
Ch 11"]
  P4["Classification
Ch 12"]

  F1["Linear Algebra · Ch 2"]
  F2["Analytic Geometry · Ch 3"]
  F3["Matrix Decomposition · Ch 4"]
  F4["Vector Calculus · Ch 5"]
  F5["Probability and Distributions · Ch 6"]
  F6["Optimization · Ch 7"]

  P1 --> ML
  P2 --> ML
  P3 --> ML
  P4 --> ML

  F1 --> P1
  F2 --> P1
  F4 --> P1
  F5 --> P1

  F1 --> P2
  F2 --> P2
  F3 --> P2
  F6 --> P2

  F1 --> P3
  F3 --> P3
  F5 --> P3
  F6 --> P3

  F1 --> P4
  F2 --> P4
  F6 --> P4

Two things are worth reading off this rather than taking on trust.

Linear algebra feeds every pillar. That is why it is Chapter 2 and why it is not optional on any route. If you read one foundation chapter, it is that one.

The pillars are not equally coupled to the foundations. Classification (SVM) leans hardest on optimisation; density estimation (GMM) leans hardest on probability; dimensionality reduction (PCA) leans hardest on matrix decomposition. So “which foundation do I need?” has a real answer once you know which pillar you are heading for.

What each foundation chapter is for

The book’s own one-line justifications, which are unusually direct:

chapter	the question it answers
2 · Linear Algebra	how do we represent numerical data as vectors, and a table of them as a matrix?
3 · Analytic Geometry	given two vectors representing two real-world objects, how do we say they are similar?
4 · Matrix Decompositions	which operations on matrices give an intuitive interpretation of the data, and more efficient learning?
5 · Vector Calculus	what is a gradient, and how do we find one?
6 · Probability and Distributions	how do we quantify noise, and express confidence in a prediction?
7 · Continuous Optimization	how do we actually find the maximum or minimum?

Notice how §3’s question is phrased: similarity. The idea is that “vectors that are similar should be predicted to have similar outputs by our machine learning algorithm”, and formalising similarity is what inner products are for. Chapter 3 is not geometry for its own sake; it exists because prediction needs a notion of “close”.

And §6’s question is phrased as noise: “we often consider data to be noisy observations of some true underlying signal”, and probability is the language for saying what noise means.

What each pillar chapter does

chapter	pillar	what is being learned	are there labels?
8	—	restates data, models and parameter estimation mathematically, and how to guard against overly optimistic evaluation	—
9	Regression	a function mapping $\mathbf{x} \in \mathbb{R}^D$ to $y \in \mathbb{R}$	yes, real-valued
10	Dimensionality reduction	a compact lower-dimensional representation of $\mathbf{x} \in \mathbb{R}^D$	no
11	Density estimation	a probability distribution describing the dataset	no
12	Classification	a function mapping $\mathbf{x}$ to a label $y$	yes, integer

The last column is the cleanest way to hold the four pillars apart, and the book uses exactly this framing. Chapters 10 and 11 have no labels at all — they model the data itself. Chapters 9 and 12 both have labels and differ in their type: real-valued for regression, integer for classification, which “requires special care” because you cannot meaningfully take a gradient with respect to a class index.

Chapters 10 and 11 also differ from each other in a way that is easy to blur: dimensionality reduction wants a low-dimensional representation of the data, and density estimation wants a distribution that describes it. Neither has labels; they want different objects.

Worked example: choosing a route

Four readers, four goals, four routes. The reasoning is what matters, not the lists.

“I want to understand backpropagation.” You need gradients and the chain rule, which is Chapter 5, which needs matrices from Chapter 2 and nothing else. Route: 0 → 2 → 5. Skip Chapter 3 and 4 entirely for now; skip probability; skip all of Part II. Three chapters, and you are done.

“I want to know why PCA works.” PCA is Chapter 10, standing on matrix decomposition (Chapter 4), which stands on analytic geometry (Chapter 3) for projections, which stands on Chapter 2. The max-variance derivation also needs a Lagrange multiplier from §7.2. Route: 2 → 3 → 4 → §7.2 → 10.

“I want to implement an SVM from scratch.” Chapter 12 leans on the dual problem and KKT conditions, which is §7.2 and §7.3.2, which need convexity, which needs gradients. Route: 2 → 3 → 5 → 7 → 12. Chapter 6 is genuinely optional here — SVMs are not probabilistic.

“I want to read modern deep learning papers.” Route: 0 → 2 → 5 → 7 → 6 → 8, and skip Part II almost entirely. Papers assume gradients, optimisation, and just enough probability to read a loss function; they rarely assume PCA or GMMs.

The pattern: name the destination, walk the arrows backwards, and read only what you land on. The book was built to allow that, and it is a much better use of time than starting at page 17 and grinding.

See it move

Pick a destination chapter with the knob. The graph highlights everything that chapter actually depends on, transitively, and greys out the rest — so you can see how much of the book any given goal requires.

sketch What does this chapter actually require? p5.js

Drag the target knob to a chapter. Highlighted boxes are its transitive prerequisites; grey boxes are chapters you can skip for that goal. The count of required chapters is shown top right.

Drag to Chapter 5 and watch nearly everything go grey — gradients need only Chapter 2. Drag to Chapter 9 and almost nothing is skippable, because regression pulls on geometry, decompositions, calculus and probability at once. That asymmetry is the practical content of §1.2.

From scratch

The dependency graph, and the “what do I actually need” query, as ten lines of Python.

reading_route.py

# The book's own dependency claims: direct prerequisites per chapter.
DEPS = {
    2: [], 3: [2], 4: [2, 3], 5: [2], 6: [2], 7: [5],
    8: [6, 7], 9: [3, 4, 5, 6, 8], 10: [3, 4, 7], 11: [4, 6, 7, 8], 12: [3, 7],
}
 
def required(ch, seen=None):
    """Transitive closure of prerequisites — everything you must read first."""
    seen = set() if seen is None else seen
    for d in DEPS[ch]:
        if d not in seen:
            seen.add(d)
            required(d, seen)
    return seen
 
def route(ch):
    """A valid reading order: prerequisites in ascending chapter number, then the target."""
    return sorted(required(ch)) + [ch]
 
for goal in (5, 7, 10, 12, 9):
    need = required(goal)
    print(f"Ch {goal:2d}: needs {len(need)} -> {sorted(need)}   skip {sorted(set(DEPS) - need - {goal})}")
 
print("\nroute to PCA        :", route(10))
print("route to SVM        :", route(12))
print("route to regression :", route(9))
 
# Which foundation appears in the most routes? A crude importance measure.
from collections import Counter
counts = Counter(c for goal in DEPS for c in required(goal))
print("\nmost-depended-on chapters:", counts.most_common(4))

text

Ch  5: needs 1 -> [2]   skip [3, 4, 6, 7, 8, 9, 10, 11, 12]
Ch  7: needs 2 -> [2, 5]   skip [3, 4, 6, 8, 9, 10, 11, 12]
Ch 10: needs 5 -> [2, 3, 4, 5, 7]   skip [6, 8, 9, 11, 12]
Ch 12: needs 4 -> [2, 3, 5, 7]   skip [4, 6, 8, 9, 10, 11]
Ch  9: needs 7 -> [2, 3, 4, 5, 6, 7, 8]   skip [10, 11, 12]
 
route to PCA        : [2, 3, 4, 5, 7, 10]
route to SVM        : [2, 3, 5, 7, 12]
route to regression : [2, 3, 4, 5, 6, 7, 8, 9]

text

most-depended-on chapters: [(2, 10), (5, 6), (3, 5), (7, 5)]

Chapter 2 appears in all ten other chapters’ prerequisite sets — it is the only universal one, and that is the quantitative version of “linear algebra feeds every pillar”. Chapter 5 is next at six, then Chapters 3 and 7 at five each. Vector calculus outranking analytic geometry is worth a moment: Chapter 5 is pulled in by everything that optimises anything, because Chapter 7 depends on it.

Note also that Chapter 6 is skippable for the SVM route and Chapter 4 is skippable for it too. That is not an approximation — support vector machines are not probabilistic and do not need a matrix factorisation.

Where the exercises live

§1.3 sets the pattern this module follows:

Part I exercises are pen and paper. The book provides them at the end of Chapters 2 to 7, and they are meant to be worked by hand. This module gives each of those chapters an Exercises and Solutions page with the problems restated, worked solutions, and a NumPy block that verifies the answer.
Part II is programming. The book ships Jupyter tutorials rather than written exercises for Chapters 8 to 12, “to explore some properties of the machine learning algorithms we discuss”. Since there are no written exercise sets for those chapters, this module’s Part II exercise pages carry problems authored in the same style.

The book is freely available at mml-book.com, which also hosts the tutorials and errata. Reading it alongside these pages is the intended use — every page here ends with a From the book box giving the exact section, definition and equation numbers so you can find the corresponding passage without hunting.

Pitfalls

Compare

	bottom-up	top-down
starts from	foundations	a problem you care about
motivation	supplied late	supplied immediately
the book’s stated cost	definitions are quickly forgotten	words you cannot define
best when	you will use this for years	you have a deadline
in this module	Ch 0 → 1 → 2 → … → 12	Ch 8 → 9, then backwards as needed
risk if done alone	giving up in Chapter 4	stalling at the first non-standard paper

pch.quizTag Check yourself

The book gives a specific objection to the bottom-up strategy. What is it?

pch.quizShowAnswer

B — Foundational concepts are not interesting by themselves, so unmotivated definitions are quickly forgotten — The objection is retention, not speed. Learning a definition with no reason to care about it means not remembering it — which is why picking a destination first helps even on a bottom-up route.
Which of the four pillars have no labels associated with their data?

pch.quizShowAnswer

B — Dimensionality reduction and density estimation — Chapters 10 and 11 model the data itself. Regression has real-valued labels and classification has integer labels, and that difference in type is why classification needs special care.
How are chapters coupled in Part II of the book?

pch.quizShowAnswer

B — They are only loosely coupled and can be read in any order, being ordered by ascending difficulty — The book says this explicitly. Part I mostly builds up; Part II is ordered by difficulty, not dependency, so you can go straight to whichever pillar you need.
Chapter 2 appears in the prerequisite set of every other chapter in the dependency graph. What follows?

pch.quizShowAnswer

B — Linear algebra is the one foundation no reading route can skip — It is the quantitative version of the book's Figure 1.1, where linear algebra sits under every pillar. Whatever you are aiming at, that chapter is on the way.

🧪 Try It Yourself

Exercise 1 – Direct prerequisites

Exercise 2 – The transitive closure

Exercise 3 – What can you skip?

Exercise 4 – The universal prerequisite

Exercise 5 – A valid reading order

Mathematics for Machine Learning §1.2 (“Two Ways to Read This Book”) is the source. The two strategies and their stated costs are quoted from that section: bottom-up risks that “the lack of motivation means that most foundational definitions are quickly forgotten”, and top-down risks “a set of words that they do not have any way of understanding”. Figure 1.1 draws the four pillars — regression, dimensionality reduction, density estimation and classification — on the foundations of linear algebra, analytic geometry, matrix decomposition, vector calculus, probability and optimisation. The section also states that Part I chapters “mostly build upon the previous ones” while Part II chapters “are only loosely coupled and can be read in any order”, ordered by ascending difficulty. The per-chapter descriptions in the tables above are the book’s own from the Part I Is about Mathematics and Part II Is about Machine Learning subsections, including the note that classification labels “are integers, which requires special care”. §1.3 covers exercises: pen and paper for Part I, Jupyter tutorials for Part II, with the book freely available at mml-book.com. The dependency graph itself is this module’s reading of those claims, not a figure in the book.

Recall card

Bottom-up’s stated cost is retention — unmotivated foundational definitions are quickly forgotten, so rigour without a destination does not stick.
Top-down’s stated cost is vocabulary without understanding — you end up fluent in words you cannot define, which holds until a paper leaves the standard recipe.
Most readers mix the two, and the book says so outright: there are more than two ways to read it.
The four pillars are regression, dimensionality reduction, density estimation and classification — Chapters 9 through 12.
Six foundations sit under them — linear algebra, analytic geometry, matrix decomposition, vector calculus, probability, and optimisation.
Linear algebra is the only universal prerequisite, appearing in every other chapter’s transitive dependency set.
The four pillars split on labels: dimensionality reduction and density estimation have none, regression has real-valued ones, classification has integer ones — and integers are why classification needs special care.
Dimensionality reduction wants a low-dimensional representation; density estimation wants a distribution — both unlabelled, different objects.
Part I mostly builds up, but Part II is only loosely coupled and ordered by difficulty rather than dependency, so any pillar can be read first.
Name the destination and walk the arrows backwards — backpropagation needs two chapters of eleven, while regression needs eight.
Part I’s exercises are pen and paper; Part II’s are Jupyter notebooks, and the book is free at mml-book.com.

Next: the foundation every route passes through — Linear Algebra.

pch.coffeeTagline

pch.coffeeCta

pch.feedbackHeading

pch.feedbackSubheading