A linear mapping is a function between vector spaces that respects the two operations that define those spaces: addition and scaling. That single requirement is astonishingly powerful — it forces every such function to be a matrix. When you understand this page, the sentence “a neural network layer is a matrix multiply” stops being a slogan and becomes obvious.

A real-life example: currency conversion

Convert a basket of currencies to dollars. Double the basket → double the dollars. Combine two baskets → the dollar values add. The conversion preserves addition and scaling, so it’s a linear map — and it’s represented by a row of exchange rates (a matrix). Rotations of an image, resizing, and a neural layer are all the same kind of object.

The definition

For vector spaces $V, W$, a mapping $\Phi : V \to W$ is linear (a vector space homomorphism) if for all $\mathbf{x}, \mathbf{y} \in V$ and $\lambda, \psi \in \mathbb{R}$:

In words: mapping a combination = combining the mappings. Equivalently, the two conditions $\Phi(\mathbf{x}+\mathbf{y}) = \Phi(\mathbf{x})+\Phi(\mathbf{y})$ and $\Phi(\lambda\mathbf{x}) = \lambda\Phi(\mathbf{x})$ both hold.

Special kinds of maps

  flowchart TD
  L["Linear map Φ : V → W"] --> INJ["Injective
Φ(x)=Φ(y) ⇒ x=y"]
  L --> SUR["Surjective
reaches all of W"]
  INJ --> BIJ["Bijective = Isomorphism
(invertible, no info lost)"]
  SUR --> BIJ
  L --> END["V → V : Endomorphism"]
  END --> AUT["bijective V → V : Automorphism"]

A bijective linear map is an isomorphism — the two spaces are “the same” for all linear purposes. A deep fact: two finite-dimensional spaces are isomorphic iff they have the same dimension. That’s why every $n$-dimensional space is “just $\mathbb{R}^n$ in disguise.”

Every linear map is a matrix

Fix an ordered basis $B = (\mathbf{b}_1,\dots,\mathbf{b}_n)$ of $V$ and $C$ of $W$. Because $\Phi$ is linear, it’s completely determined by what it does to the basis vectors. Collect the coordinates of $\Phi(\mathbf{b}_j)$ into the columns of a matrix $A_\Phi$ — the transformation matrix. Then mapping a vector is just a matrix-vector product on its coordinates:

where $\hat{\mathbf{x}}$ and $\hat{\mathbf{y}}$ are the coordinate vectors of $\mathbf{x}$ and $\Phi(\mathbf{x})$. The columns of the matrix are the images of the basis vectors — the exact fact you saw the amber/violet arrows demonstrate on the Matrices page.

Image and kernel

Two subspaces capture everything about a map’s behavior:

• The image (or range) $\text{Im}(\Phi) = \{\Phi(\mathbf{x}) : \mathbf{x}\in V\}$ — all the outputs you can reach. For $\Phi(\mathbf{x}) = A\mathbf{x}$, this is the column space of $A$: $\text{Im}(\Phi) = \text{span}[\text{columns of } A]$.
• The kernel (or null space) $\ker(\Phi) = \{\mathbf{x} : \Phi(\mathbf{x}) = \mathbf{0}\}$ — everything the map crushes to zero.

$\Phi$ is injective iff $\ker(\Phi) = \{\mathbf{0}\}$ — nothing gets crushed, so nothing collides.

Watch a map squash the plane

Take the singular map $\Phi(\mathbf{x}) = A\mathbf{x}$ with $A = \begin{bmatrix}1&2\\2&4\end{bmatrix}$. Its columns are collinear, so the whole plane gets squashed onto a single line (the image). Every input along the perpendicular kernel direction is mapped straight to the origin. Blue dots are inputs; amber dots are where they land — notice they all pile onto the amber image line:

The picture is the rank-nullity theorem: the input plane is 2-dimensional, the image is 1-dimensional, and the kernel is 1-dimensional. $1 + 1 = 2$.

The rank-nullity theorem

For any linear map $\Phi : V \to W$ on a finite-dimensional $V$:

Also called the fundamental theorem of linear mappings. It says dimensions are conserved: whatever the map crushes (kernel) plus whatever it preserves (image) always adds back up to the input dimension. Since $\dim(\text{Im}\,\Phi) = \text{rk}(A)$, this ties the whole chapter together.

Basis change (a quick look)

The same linear map has different matrices in different bases. If $S$ and $T$ are the change-of-basis matrices in $V$ and $W$, the transformation matrix transforms as

Matrices related this way are called equivalent; when $V = W$ and $S = T$, they’re similar ($\tilde{A} = S^{-1}AS$). Choosing a clever basis can make a map’s matrix diagonal — the entire point of eigendecomposition (Chapter 4) and PCA (Chapter 10).

NumPy: image, kernel, rank-nullity

import numpy as np
 
A = np.array([[1.0, 2.0],
              [2.0, 4.0]])            # rank-1: squashes the plane
 
n = A.shape[1]
rank = np.linalg.matrix_rank(A)       # dim of image
 
# Kernel basis from the SVD: right-singular vectors with ~zero singular value
u, s, vh = np.linalg.svd(A)
ker = vh[rank:].T                     # columns span the kernel
dim_ker = n - rank
 
print("rank  = dim(image) :", rank)
print("dim(kernel)        :", dim_ker)
print("rank-nullity check :", rank + dim_ker, "== dim(V) = 2")
print("a kernel vector    :", np.round(ker[:, 0], 3), "-> A@it =", np.round(A @ ker[:, 0], 6))

import numpy as np
 
A = np.array([[1.0, 2.0],
              [2.0, 4.0]])            # rank-1: squashes the plane
 
n = A.shape[1]
rank = np.linalg.matrix_rank(A)       # dim of image
 
# Kernel basis from the SVD: right-singular vectors with ~zero singular value
u, s, vh = np.linalg.svd(A)
ker = vh[rank:].T                     # columns span the kernel
dim_ker = n - rank
 
print("rank  = dim(image) :", rank)
print("dim(kernel)        :", dim_ker)
print("rank-nullity check :", rank + dim_ker, "== dim(V) = 2")
print("a kernel vector    :", np.round(ker[:, 0], 3), "-> A@it =", np.round(A @ ker[:, 0], 6))

rank  = dim(image) : 1
dim(kernel)        : 1
rank-nullity check : 2 == dim(V) = 2
a kernel vector    : [ 0.894 -0.447] -> A@it = [ 0. -0.]

rank  = dim(image) : 1
dim(kernel)        : 1
rank-nullity check : 2 == dim(V) = 2
a kernel vector    : [ 0.894 -0.447] -> A@it = [ 0. -0.]

Why this matters for ML

• A dense neural-network layer is $\mathbf{y} = W\mathbf{x} + \mathbf{b}$ — a linear map plus a shift. The weights are the transformation matrix.
• Rank-nullity explains information loss: a layer that maps to a lower-dimensional image is literally throwing away $\dim(\ker)$ directions of your data.
• Basis change / diagonalization is what makes PCA, whitening, and spectral methods work — pick the basis where the map is simplest.

🧪 Try It Yourself

Exercise 1 – Test linearity

Exercise 2 – Verify rank-nullity

Exercise 3 – Find a kernel vector

Recap

• A linear map preserves addition and scaling: $\Phi(\lambda\mathbf{x}+\psi\mathbf{y}) = \lambda\Phi(\mathbf{x})+\psi\Phi(\mathbf{y})$.
• Every linear map between finite-dimensional spaces is a matrix (columns = images of basis vectors); mapping = matrix-vector product on coordinates.
• The image is the column space; the kernel is what maps to $\mathbf{0}$; injective $\iff$ kernel is trivial.
• Rank-nullity: $\dim(\ker\Phi) + \dim(\text{Im}\,\Phi) = \dim(V)$.
• Basis change $\tilde{A} = T^{-1}AS$ lets you pick the coordinate system where the map is simplest — the seed of PCA and eigendecomposition.

Next: what happens when the geometry is shifted off the origin — Affine Spaces.