Once you have an inner product, length and distance come for free. The length of a vector is the square root of its inner product with itself; the distance between two vectors is the length of their difference. These two quantities power k-NN, clustering, and every “how close are these?” question in machine learning.

A real-life example: nearest neighbors

A recommendation engine represents each song as a vector of audio features. To recommend “songs like this one,” it finds the vectors with the smallest distance to your current track. Change the notion of distance and you change the recommendations — which is why the length/distance machinery below is not academic, it’s the product.

Length from an inner product

Any inner product induces a norm:

With the dot product this is the familiar Euclidean length $\sqrt{\mathbf{x}^\top\mathbf{x}}$. Not every norm comes from an inner product (the Manhattan norm doesn’t) — but the ones that do inherit a beautiful bonus:

The Cauchy-Schwarz inequality

The inner product can never exceed the product of the lengths. This single inequality is what makes the angle on the next page well-defined (it keeps $\cos\omega$ inside $[-1, 1]$).

Distance and metrics

The distance between two vectors is the length of their difference:

With the dot product this is Euclidean distance. A distance function that satisfies three axioms is called a metric:

  flowchart TD
  M["A metric d(x, y)"] --> P["positive definite
d ≥ 0, and d=0 ⇔ x=y"]
  M --> S["symmetric
d(x,y) = d(y,x)"]
  M --> T["triangle inequality
d(x,z) ≤ d(x,y) + d(y,z)"]

The triangle inequality, live

The triangle inequality says a detour through a third point $\mathbf{y}$ is never shorter than going straight from $\mathbf{x}$ to $\mathbf{z}$. Drag the middle point around (it orbits) and watch the direct distance stay $\le$ the two-leg path:

NumPy

import numpy as np
 
x = np.array([1.0, 1.0])
y = np.array([4.0, 5.0])
 
# length from the (dot-product) inner product
print("‖x‖ =", np.sqrt(x @ x))              # √2 ≈ 1.414
 
# Euclidean distance
print("d(x, y) =", np.linalg.norm(x - y))   # 5.0
 
# Cauchy-Schwarz: |⟨x,y⟩| ≤ ‖x‖‖y‖
lhs = abs(x @ y)
rhs = np.linalg.norm(x) * np.linalg.norm(y)
print(f"Cauchy-Schwarz: {lhs:.3f} ≤ {rhs:.3f} -> {lhs <= rhs}")

import numpy as np
 
x = np.array([1.0, 1.0])
y = np.array([4.0, 5.0])
 
# length from the (dot-product) inner product
print("‖x‖ =", np.sqrt(x @ x))              # √2 ≈ 1.414
 
# Euclidean distance
print("d(x, y) =", np.linalg.norm(x - y))   # 5.0
 
# Cauchy-Schwarz: |⟨x,y⟩| ≤ ‖x‖‖y‖
lhs = abs(x @ y)
rhs = np.linalg.norm(x) * np.linalg.norm(y)
print(f"Cauchy-Schwarz: {lhs:.3f} ≤ {rhs:.3f} -> {lhs <= rhs}")

‖x‖ = 1.4142135623730951
d(x, y) = 5.0
Cauchy-Schwarz: 9.000 ≤ 9.220 -> True

‖x‖ = 1.4142135623730951
d(x, y) = 5.0
Cauchy-Schwarz: 9.000 ≤ 9.220 -> True

Why this matters for ML

• k-Nearest Neighbors classifies a point by the labels of the closest training points — “closest” is a distance.
• Clustering (k-means) minimizes within-cluster squared distances.
• Loss functions like mean squared error are squared Euclidean distances between predictions and targets.

🧪 Try It Yourself

Exercise 1 – Length and distance

Exercise 2 – Nearest neighbor

Exercise 3 – Distance is symmetric

Recap

• An inner product induces a length $\lVert \mathbf{x}\rVert = \sqrt{\langle \mathbf{x},\mathbf{x}\rangle}$ and a distance $d(\mathbf{x},\mathbf{y}) = \lVert \mathbf{x}-\mathbf{y}\rVert$.
• Cauchy-Schwarz ($|\langle \mathbf{x},\mathbf{y}\rangle| \le \lVert\mathbf{x}\rVert\lVert\mathbf{y}\rVert$) keeps angles well-defined.
• A metric is positive-definite, symmetric, and obeys the triangle inequality.
• Distance powers k-NN, clustering, and squared-error losses.

Next: the same inner product measures the angle between vectors.