So far we’ve computed with vectors. Now we zoom out and ask: what is the arena they live in? That arena is a vector space — a set where “add two things” and “scale a thing” always land you back inside the set. Getting this abstraction is what lets the same math describe arrows, images, audio, and word embeddings all at once.

A real-life example: RGB color space

Every screen color is a vector $(r, g, b)$ with each component in $[0, 255]$. Mix two colors (add) or dim a color (scale) and you get… another color. Colors are closed under mixing and dimming — they form (essentially) a vector space. That “you never leave the set” property is the heart of the definition.

Building up: groups first

A group $(\mathcal{G}, \otimes)$ is a set with one operation obeying four rules: closure (the result stays in the set), associativity, a neutral element $e$, and an inverse for every element. If the operation also commutes, the group is Abelian.

$(\mathbb{Z}, +)$ is a group (neutral $0$, inverse $-x$). $(\mathbb{N}_0, +)$ is not — there’s no inverse for $3$ inside the naturals. Groups are the minimal skeleton; a vector space adds scaling on top.

The vector-space axioms

A real vector space $V = (\mathcal{V}, +, \cdot)$ is a set $\mathcal{V}$ with two operations:

such that:

1. $(\mathcal{V}, +)$ is an Abelian group (add vectors, get a vector; there’s a zero vector $\mathbf{0}$ and negatives).
2. Distributivity: $\lambda\cdot(\mathbf{x}+\mathbf{y}) = \lambda\mathbf{x} + \lambda\mathbf{y}$ and $(\lambda+\psi)\cdot\mathbf{x} = \lambda\mathbf{x} + \psi\mathbf{x}$.
3. Associativity of scaling: $\lambda\cdot(\psi\cdot\mathbf{x}) = (\lambda\psi)\cdot\mathbf{x}$.
4. Scaling identity: $1\cdot\mathbf{x} = \mathbf{x}$.

The elements of $V$ are vectors; the $\lambda \in \mathbb{R}$ are scalars. Two workhorse examples:

• $\mathbb{R}^n$ — column vectors of $n$ reals, added and scaled component-wise.
• $\mathbb{R}^{m\times n}$ — all $m\times n$ matrices, added and scaled entry-wise.

Subspaces: a vector space inside a vector space

A subspace $U \subseteq V$ is a subset that is itself a vector space under the same operations. You don’t have to recheck all the axioms — inherited from $V$, they hold automatically. You only have to check three things:

1. $U$ is non-empty, and in particular contains $\mathbf{0}$.
2. Closed under addition: $\mathbf{x}, \mathbf{y} \in U \Rightarrow \mathbf{x}+\mathbf{y} \in U$.
3. Closed under scaling: $\lambda \in \mathbb{R},\, \mathbf{x} \in U \Rightarrow \lambda\mathbf{x} \in U$.

  flowchart TD
  Q["Is U a subspace of V?"] --> Z["Contains 0?"]
  Z -->|no| NO["NOT a subspace"]
  Z -->|yes| ADD["Closed under +?"]
  ADD -->|no| NO
  ADD -->|yes| SC["Closed under scaling?"]
  SC -->|no| NO
  SC -->|yes| YES["It's a subspace"]

The interactive test

The single most common trap: a line that misses the origin is not a subspace. Watch two vectors that both live on a set get added together. On a line through the origin (green), the sum stays on the line — closed. On a line offset from the origin (red), the sum flies off the set — not closed, not a subspace:

Both lines are perfectly good geometric objects — but only the one containing $\mathbf{0}$ is a subspace. This is exactly why, later, a regression line (which is offset) lives in an affine space, not a subspace.

The ML payoff: solution sets

Two facts connect this straight back to the previous pages:

• The solution set of a homogeneous system $A\mathbf{x} = \mathbf{0}$ is a subspace of $\mathbb{R}^n$ (it always contains $\mathbf{0}$ and is closed). This subspace is the null space.
• The solution set of an inhomogeneous system $A\mathbf{x} = \mathbf{b}$ with $\mathbf{b}\neq\mathbf{0}$ is not a subspace — it’s a shifted (affine) set, because it doesn’t contain $\mathbf{0}$.

NumPy: checking closure numerically

import numpy as np
 
def on_line(p, direction, intercept):
    # is point p on the line {t*direction + intercept}?
    dx, dy = direction
    return np.isclose(p[1], (dy / dx) * p[0] + intercept)
 
direction = (1.4, 0.8)
 
# Case A: line through the origin (intercept 0)
u = np.array([1.4, 0.8])       # t = 1
v = np.array([-0.7, -0.4])     # t = -0.5
print("through origin, u+v on set?", on_line(u + v, direction, 0.0))
 
# Case B: line shifted up by 1.6
u2 = np.array([1.4, 0.8 + 1.6])
v2 = np.array([-0.7, -0.4 + 1.6])
print("shifted, u+v on set?      ", on_line(u2 + v2, direction, 1.6))

import numpy as np
 
def on_line(p, direction, intercept):
    # is point p on the line {t*direction + intercept}?
    dx, dy = direction
    return np.isclose(p[1], (dy / dx) * p[0] + intercept)
 
direction = (1.4, 0.8)
 
# Case A: line through the origin (intercept 0)
u = np.array([1.4, 0.8])       # t = 1
v = np.array([-0.7, -0.4])     # t = -0.5
print("through origin, u+v on set?", on_line(u + v, direction, 0.0))
 
# Case B: line shifted up by 1.6
u2 = np.array([1.4, 0.8 + 1.6])
v2 = np.array([-0.7, -0.4 + 1.6])
print("shifted, u+v on set?      ", on_line(u2 + v2, direction, 1.6))

through origin, u+v on set? True
shifted, u+v on set?       False

through origin, u+v on set? True
shifted, u+v on set?       False

Why this matters for ML

• Feature spaces are vector spaces: your whole dataset lives in $\mathbb{R}^n$.
• PCA (Chapter 10) works by finding a low-dimensional subspace that captures most of the data’s variance — compression is literally choosing a good subspace.
• Knowing that homogeneous solutions form a subspace (but inhomogeneous ones don’t) is what makes the “particular + null space” decomposition from the last page work.

🧪 Try It Yourself

Exercise 1 – The zero-vector test

Exercise 2 – Closure under addition

Exercise 3 – Null space is a subspace

Recap

• A vector space is a set closed under addition and scalar multiplication, obeying the group + distributivity axioms.
• $\mathbb{R}^n$ and $\mathbb{R}^{m\times n}$ are the everyday examples; so are polynomials and functions.
• A subspace must contain $\mathbf{0}$ and be closed under $+$ and scaling — a line through the origin qualifies, a shifted line does not.
• Homogeneous solution sets are subspaces (the null space); inhomogeneous ones are affine.

Next: inside a space, which vectors carry genuinely new directions? — Linear Independence.