Given a subspace, the orthogonal complement is everything perpendicular to it. It’s the normal direction of a plane, the “leftover” space a projection discards, and the reason any vector can be split cleanly into a part inside a subspace and a part outside it. That split is the mathematical heart of PCA and least-squares residuals.

A real-life example: a wall and its normal

Stand a flat panel (a 2-D plane) in a room (3-D space). Every direction lying in the panel forms the subspace $U$. The single direction sticking straight out of the panel — its normal — is the orthogonal complement $U^\perp$. Together they describe the whole room: any arrow in the room is “some slide along the panel” plus “some poke through it.”

Definition

For a $D$-dimensional space $V$ and an $M$-dimensional subspace $U \subseteq V$, the orthogonal complement $U^\perp$ is the set of all vectors orthogonal to every vector in $U$:

Key facts:

• $U^\perp$ is a subspace of dimension $D - M$.
• $U \cap U^\perp = \{\mathbf{0}\}$ — they share only the origin.
• Every $\mathbf{x} \in V$ decomposes uniquely as $\mathbf{x} = \mathbf{x}_U + \mathbf{x}_{U^\perp}$ with $\mathbf{x}_U \in U$ and $\mathbf{x}_{U^\perp} \in U^\perp$.

  flowchart TD
  V["Space V (dim D)"] --> U["Subspace U (dim M)"]
  V --> UP["Complement U⊥ (dim D−M)"]
  U --> DEC["every x = x_U + x_U⊥"]
  UP --> DEC
  DEC --> NORM["for a plane in 3-D, U⊥ is its normal vector w"]

For a plane $U$ in 3-D, $U^\perp$ is one-dimensional — spanned by the plane’s normal vector $\mathbf{w}$. This is how a hyperplane is specified in $n$ dimensions: give its normal.

Splitting a vector, live

In 2-D, take a line $U$ through the origin. Its orthogonal complement $U^\perp$ is the perpendicular line. Any vector $\mathbf{x}$ splits into a piece along $U$ (green) and a piece along $U^\perp$ (violet) — and those two pieces add back to $\mathbf{x}$. Watch $\mathbf{x}$ rotate while its two orthogonal components track it:

The two colored components are always at right angles and always sum to $\mathbf{x}$ — the unique $U$/$U^\perp$ decomposition in action.

NumPy

import numpy as np
 
# U is the line spanned by d (a 1-D subspace of R^2)
d = np.array([1.0, 1.0]) / np.sqrt(2)      # unit direction of U
x = np.array([3.0, 1.0])
 
# component of x in U, and the rest (which lies in U-perp)
x_U = (x @ d) * d
x_perp = x - x_U
 
print("x_U      :", np.round(x_U, 4))
print("x_perp   :", np.round(x_perp, 4))
print("sum == x :", np.allclose(x_U + x_perp, x))
print("orthogonal parts:", np.isclose(x_U @ x_perp, 0))
 
# In 3-D, the normal of a plane spans its 1-D orthogonal complement
w = np.array([0.0, 0.0, 1.0])              # normal to the xy-plane
in_plane = np.array([2.0, -3.0, 0.0])
print("normal ⟂ plane vector:", np.isclose(w @ in_plane, 0))

import numpy as np
 
# U is the line spanned by d (a 1-D subspace of R^2)
d = np.array([1.0, 1.0]) / np.sqrt(2)      # unit direction of U
x = np.array([3.0, 1.0])
 
# component of x in U, and the rest (which lies in U-perp)
x_U = (x @ d) * d
x_perp = x - x_U
 
print("x_U      :", np.round(x_U, 4))
print("x_perp   :", np.round(x_perp, 4))
print("sum == x :", np.allclose(x_U + x_perp, x))
print("orthogonal parts:", np.isclose(x_U @ x_perp, 0))
 
# In 3-D, the normal of a plane spans its 1-D orthogonal complement
w = np.array([0.0, 0.0, 1.0])              # normal to the xy-plane
in_plane = np.array([2.0, -3.0, 0.0])
print("normal ⟂ plane vector:", np.isclose(w @ in_plane, 0))

x_U      : [2. 2.]
x_perp   : [ 1. -1.]
sum == x : True
orthogonal parts: True
normal ⟂ plane vector: True

x_U      : [2. 2.]
x_perp   : [ 1. -1.]
sum == x : True
orthogonal parts: True
normal ⟂ plane vector: True

Why this matters for ML

• PCA keeps a subspace $U$ of top directions and discards its complement $U^\perp$; the discarded part is exactly the reconstruction error.
• Least-squares residuals live in the orthogonal complement of the column space — the fit is in $U$, the error is in $U^\perp$.
• Hyperplane classifiers (SVMs) are specified by their normal vector, a basis of the boundary’s orthogonal complement.

🧪 Try It Yourself

Exercise 1 – Split a vector

Exercise 2 – The parts are orthogonal

Exercise 3 – A plane’s normal

Recap

• The orthogonal complement $U^\perp$ is everything perpendicular to a subspace $U$; it has dimension $D - M$.
• Every vector splits uniquely into a $U$-part plus a $U^\perp$-part.
• For a plane, $U^\perp$ is spanned by the normal vector — how hyperplanes are specified.
• In ML, the complement is the residual/discarded space in PCA and least squares.

Next: the same orthogonality idea, applied to functions.