Everything so far treated a vector as a finite list of numbers. But a function is just a vector with infinitely many “entries” — one value for each input $x$. The inner product extends to this setting by turning the sum into an integral, and suddenly two functions can be orthogonal. That idea is the entire foundation of Fourier analysis and signal processing.

A real-life example: separating audio frequencies

A recorded chord is one wiggly signal, but your ear (and an equalizer) can pull out the individual notes. That works because the pure tones (sines and cosines of different frequencies) are orthogonal functions — each carries independent information. Measuring “how much of frequency $f$ is in the signal” is an inner product of the signal with $\cos(f x)$.

From sum to integral

For finite vectors, the dot product sums products component-by-component: $\mathbf{x}^\top\mathbf{y} = \sum_i x_i y_i$. For functions $u, v : \mathbb{R} \to \mathbb{R}$, the components form a continuum, so the sum becomes an integral over an interval $[a, b]$:

All the usual machinery carries over: the norm of a function is $\lVert u\rVert = \sqrt{\langle u, u\rangle}$, and two functions are orthogonal when their inner product is zero.

Orthogonal functions: sin and cos

The classic example: on $[-\pi, \pi]$, $\sin(x)$ and $\cos(x)$ are orthogonal, because their product $\sin(x)\cos(x)$ is an odd function — the positive area on one side exactly cancels the negative area on the other, so the integral is zero.

Watch the cancellation: the product curve’s shaded area above the axis (green) is exactly matched by the area below (red), summing to zero:

Fourier: a whole orthogonal family

It’s not just sin and cos. On $[-\pi, \pi]$ the entire collection

is mutually orthogonal. Projecting a function onto this orthogonal family is exactly a Fourier series — decomposing any signal into pure frequencies. Orthogonality is what makes each Fourier coefficient independent and easy to compute (just an inner product).

  flowchart LR
  V["Finite vectors
⟨x,y⟩ = Σ xᵢyᵢ"] -->|"sum → integral"| F["Functions
⟨u,v⟩ = ∫ u(x)v(x) dx"]
  F --> O["orthogonal functions
∫ u·v = 0"]
  O --> FR["Fourier series
project onto {1, cos x, cos 2x, …}"]

NumPy: verify orthogonality numerically

import numpy as np
 
# Approximate ⟨u, v⟩ = ∫_{-π}^{π} u(x) v(x) dx with the trapezoid rule.
x = np.linspace(-np.pi, np.pi, 2001)
 
def inner(u, v):
    return np.trapz(u * v, x)
 
sin, cos = np.sin(x), np.cos(x)
print("⟨sin, cos⟩ =", round(inner(sin, cos), 6))     # ~0 → orthogonal
print("⟨sin, sin⟩ =", round(inner(sin, sin), 6))     # ~π → nonzero length²
 
# cos(x) and cos(2x) are also orthogonal on [-π, π]
print("⟨cos x, cos 2x⟩ =", round(inner(cos, np.cos(2*x)), 6))   # ~0

import numpy as np
 
# Approximate ⟨u, v⟩ = ∫_{-π}^{π} u(x) v(x) dx with the trapezoid rule.
x = np.linspace(-np.pi, np.pi, 2001)
 
def inner(u, v):
    return np.trapz(u * v, x)
 
sin, cos = np.sin(x), np.cos(x)
print("⟨sin, cos⟩ =", round(inner(sin, cos), 6))     # ~0 → orthogonal
print("⟨sin, sin⟩ =", round(inner(sin, sin), 6))     # ~π → nonzero length²
 
# cos(x) and cos(2x) are also orthogonal on [-π, π]
print("⟨cos x, cos 2x⟩ =", round(inner(cos, np.cos(2*x)), 6))   # ~0

⟨sin, cos⟩ = 0.0
⟨sin, sin⟩ = 3.141593
⟨cos x, cos 2x⟩ = -0.0

⟨sin, cos⟩ = 0.0
⟨sin, sin⟩ = 3.141593
⟨cos x, cos 2x⟩ = -0.0

Why this matters for ML

• Fourier / spectral features decompose signals (audio, EEG, time series) into orthogonal frequency components — a standard feature-engineering step.
• Gaussian processes and kernel methods work in function spaces where these inner products define similarity between functions.
• Signal compression (JPEG, MP3) keeps a few large orthogonal-basis coefficients and discards the rest — the same “project and truncate” idea as PCA.

🧪 Try It Yourself

Exercise 1 – Function inner product by integration

Exercise 2 – Norm of a function

Exercise 3 – Two cosines are orthogonal

Recap

• Functions are vectors with infinitely many components; the inner product becomes an integral $\langle u, v\rangle = \int_a^b u(x)v(x)\,dx$.
• Functions are orthogonal when that integral is zero — e.g. $\sin$ and $\cos$ on $[-\pi,\pi]$.
• The orthogonal family $\{1, \cos x, \cos 2x, \dots\}$ underlies the Fourier series.
• This powers spectral features, kernel methods, and signal compression in ML.

Next: the payoff of all this orthogonality — orthogonal projections.