A basis (Chapter 2) gives you a coordinate system. An orthonormal basis (ONB) gives you the best possible coordinate system: its axes are mutually perpendicular and each has unit length. In such a basis, coordinates are trivial to compute, projections are cheap, and numerical routines are stable — which is why PCA, QR, and SVD all produce orthonormal bases.

A real-life example: good coordinate axes

Think of standard map coordinates: “east” and “north” are perpendicular and measured in the same unit (kilometers). That’s an orthonormal basis, and it’s why reading a position off the map is easy — you just measure along each axis independently. A skewed, differently-scaled grid would make the same task a headache. ONBs are the “east/north” of any vector space.

Definition

A basis $\{\mathbf{b}_1, \dots, \mathbf{b}_n\}$ of an $n$-dimensional space is an orthonormal basis if

The first condition says the vectors are mutually orthogonal; the second says each has unit length. Drop the unit-length requirement and you have merely an orthogonal basis. The canonical basis $\{\mathbf{e}_1, \dots, \mathbf{e}_n\}$ is the standard ONB under the dot product.

Why ONBs are so convenient

In an orthonormal basis, the coordinates of any vector are just inner products:

No solving a linear system, no matrix inverse — each coordinate is a single dot product. This is the shortcut that makes projections onto an ONB nearly free (a fact we’ll cash in on the projections page).

Building one: the Gram-Schmidt process

Given any basis of non-orthogonal vectors, Gram-Schmidt turns it into an orthonormal one, one vector at a time:

1. Normalize the first vector: $\mathbf{u}_1 = \mathbf{b}_1 / \lVert\mathbf{b}_1\rVert$.
2. Subtract from $\mathbf{b}_2$ its component along $\mathbf{u}_1$, then normalize: $\mathbf{u}_2 = \dfrac{\mathbf{b}_2 - \langle \mathbf{b}_2, \mathbf{u}_1\rangle\,\mathbf{u}_1}{\lVert \cdots \rVert}$.
3. Repeat, always subtracting off the parts already covered.

Watch it happen on two vectors: the blue vector $\mathbf{b}_2$ rotates, and Gram-Schmidt produces the green $\mathbf{u}_2$ — always perpendicular to the amber $\mathbf{u}_1$ — by removing the shadow (grey) that $\mathbf{b}_2$ casts along $\mathbf{u}_1$:

No matter how $\mathbf{b}_2$ points, the green output is perpendicular to the amber $\mathbf{u}_1$ and unit length — a fresh orthonormal basis.

NumPy: Gram-Schmidt and QR

import numpy as np
 
def gram_schmidt(vectors):
    basis = []
    for v in vectors:
        w = v - sum((v @ u) * u for u in basis)   # remove components along earlier vectors
        if np.linalg.norm(w) > 1e-10:
            basis.append(w / np.linalg.norm(w))    # normalize
    return np.array(basis)
 
vs = [np.array([2.0, 0.0]), np.array([1.0, 1.0])]
Q = gram_schmidt(vs)
print("orthonormal basis:\n", np.round(Q, 4))
print("QᵀQ = I:", np.allclose(Q @ Q.T, np.eye(2)))
 
# NumPy's built-in: QR decomposition gives an orthonormal Q directly
A = np.array([[2.0, 1.0], [0.0, 1.0]])
Q2, R = np.linalg.qr(A)
print("QR's Q orthonormal:", np.allclose(Q2.T @ Q2, np.eye(2)))

import numpy as np
 
def gram_schmidt(vectors):
    basis = []
    for v in vectors:
        w = v - sum((v @ u) * u for u in basis)   # remove components along earlier vectors
        if np.linalg.norm(w) > 1e-10:
            basis.append(w / np.linalg.norm(w))    # normalize
    return np.array(basis)
 
vs = [np.array([2.0, 0.0]), np.array([1.0, 1.0])]
Q = gram_schmidt(vs)
print("orthonormal basis:\n", np.round(Q, 4))
print("QᵀQ = I:", np.allclose(Q @ Q.T, np.eye(2)))
 
# NumPy's built-in: QR decomposition gives an orthonormal Q directly
A = np.array([[2.0, 1.0], [0.0, 1.0]])
Q2, R = np.linalg.qr(A)
print("QR's Q orthonormal:", np.allclose(Q2.T @ Q2, np.eye(2)))

orthonormal basis:
 [[1. 0.]
 [0. 1.]]
QᵀQ = I: True
QR's Q orthonormal: True

orthonormal basis:
 [[1. 0.]
 [0. 1.]]
QᵀQ = I: True
QR's Q orthonormal: True

Why this matters for ML

• PCA returns an orthonormal basis of “principal directions”; projecting onto the top few compresses data with minimal loss.
• QR / SVD rely on orthonormal bases for numerically stable solutions to least-squares and eigenvalue problems.
• Orthonormal weight init/constraints keep signal magnitudes stable across deep-network layers.

🧪 Try It Yourself

Exercise 1 – Check an orthonormal basis

Exercise 2 – Coordinates are just dot products

Exercise 3 – Orthogonalize with Gram-Schmidt

Recap

• An orthonormal basis has mutually orthogonal, unit-length vectors.
• In an ONB, coordinates are just inner products — no linear solve, no inverse.
• Gram-Schmidt builds an ONB from any basis by removing overlaps and normalizing; NumPy’s qrqr does it for you.
• ONBs power PCA, QR/SVD, and stable deep-net weights.

Next: everything perpendicular to a subspace — the orthogonal complement.