Chapter 2 gave us vectors and the operations that combine them. But it never asked the geometric questions: how long is this vector? how far apart are two vectors? what’s the angle between them? Chapter 3 answers all of these by introducing one powerful object — the inner product — and building lengths, distances, angles, orthogonality, and projections on top of it.

This is the layer that turns algebra into geometry, and geometry is where machine learning lives: “similar” documents are vectors with a small angle, a good recommendation is a nearby vector, PCA projects data onto a subspace, and an SVM finds the widest gap around a separating hyperplane.

The concept map

Everything in this chapter flows from the inner product.

  flowchart TD
  IP["Inner product
⟨x, y⟩"]
  N["Norm / length
‖x‖ = √⟨x,x⟩"]
  D["Distance
d(x,y) = ‖x − y‖"]
  A["Angle
cos ω = ⟨x,y⟩ / (‖x‖‖y‖)"]
  O["Orthogonality
⟨x, y⟩ = 0"]
  ONB["Orthonormal basis"]
  OC["Orthogonal complement"]
  P["Orthogonal projection"]
  R["Rotations"]

  IP --> N
  N --> D
  IP --> A
  A --> O
  O --> ONB
  O --> OC
  ONB --> P
  OC --> P
  P -.-> PCA["PCA / regression (Ch 9, 10)"]
  O -.-> SVM["SVM boundaries (Ch 12)"]
  R -.-> ROT["data augmentation, graphics"]

The through-line

1. Norms — how to measure the length of a vector (and why there’s more than one way).
2. Inner Products — the generalized dot product that unlocks all the geometry.
3. Lengths and Distances — length from an inner product, and the distance between two vectors.
4. Angles and Orthogonality — measuring the angle between vectors; when they’re at right angles.
5. Orthonormal Basis — a basis of perpendicular unit vectors, the nicest coordinate system there is.
6. Orthogonal Complement — everything perpendicular to a subspace; the normal vector of a plane.
7. Inner Product of Functions — the same idea applied to functions (the seed of Fourier analysis).
8. Orthogonal Projections — dropping data onto a subspace with least error; the engine of PCA and least squares.
9. Rotations — length- and angle-preserving transformations.

Where each idea shows up in ML

Prerequisites

Everything from Chapter 2 — Linear Algebra: vectors, matrices, subspaces, basis, and linear maps. If “column space” or “basis” feels shaky, skim that chapter’s overview first.

Start with Norms.