The eigendecomposition writes a square matrix as $A = PDP^{-1}$: a change into the eigenbasis ($P^{-1}$), a pure scaling along each eigen-axis ($D$, diagonal), and a change back ($P$). In this coordinate system the matrix is diagonal — the simplest possible form. That’s what makes matrix powers trivial, PCA computable, and the geometry of a transformation obvious.

A real-life example: predicting a population years ahead

A wildlife model updates a population vector each year by multiplying by a transition matrix $A$. To predict 50 years out you’d need $A^{50}$ — 50 matrix multiplies. But if $A = PDP^{-1}$, then $A^{50} = PD^{50}P^{-1}$, and $D^{50}$ is just each diagonal entry raised to the 50th power. The eigendecomposition turns “multiply 50 times” into “raise a few numbers to a power.” Long-term behavior is dictated by the largest eigenvalue.

Diagonal matrices are easy

A diagonal matrix $D$ has zeros off the diagonal. Everything about it is cheap: its determinant is the product of the diagonal, its power $D^k$ raises each diagonal entry to the $k$, and its inverse is the reciprocal of each entry. The goal of diagonalization is to transform a general matrix into a diagonal one by choosing the right basis.

The eigendecomposition

A matrix $A \in \mathbb{R}^{n\times n}$ is diagonalizable if it has $n$ linearly independent eigenvectors. Collect them as the columns of $P$ and the eigenvalues on the diagonal of $D$; then

If $A$ is symmetric, the spectral theorem gives an orthonormal eigenbasis, so $P$ is orthogonal and $P^{-1} = P^\top$:

The geometry: rotate, scale, rotate back

For a symmetric $A$, applying it to the unit circle produces an ellipse whose axes point along the eigenvectors and whose half-lengths are the eigenvalues. That’s the eigendecomposition made visible: $P^\top$ rotates the eigenvectors onto the standard axes, $D$ stretches by the eigenvalues, and $P$ rotates back. Watch the circle become an ellipse aligned with the amber eigen-axes:

Matrix powers and other payoffs

Because $A^k = (PDP^{-1})^k = PD^kP^{-1}$, powers are cheap. So are:

• Determinant: $\det(A) = \det(D) = \prod_i \lambda_i$.
• Matrix functions: $e^A = Pe^DP^{-1}$, $\sqrt{A} = PD^{1/2}P^{-1}$ — apply the function to the eigenvalues.

  flowchart LR
  X["x"] -->|"P⁻¹: to eigenbasis"| C1["eigen-coords"]
  C1 -->|"D: scale by λᵢ"| C2["scaled coords"]
  C2 -->|"P: back to standard"| Y["A x"]

Not every matrix is diagonalizable — defective matrices (too few independent eigenvectors) aren’t. But every symmetric matrix always is, which is exactly the case that matters most in ML.

NumPy

import numpy as np
 
A = np.array([[2.0, 1.0],
              [1.0, 2.0]])       # symmetric
 
vals, P = np.linalg.eig(A)
D = np.diag(vals)
 
# reconstruct A = P D P^-1
A_rebuilt = P @ D @ np.linalg.inv(P)
print("A = P D P⁻¹:", np.allclose(A, A_rebuilt))
 
# matrix power the cheap way: A^5 = P D^5 P^-1
A5_fast = P @ np.diag(vals**5) @ np.linalg.inv(P)
A5_slow = np.linalg.matrix_power(A, 5)
print("A⁵ matches:", np.allclose(A5_fast, A5_slow))
 
# symmetric ⇒ orthogonal P, so P^-1 = P^T
w, Q = np.linalg.eigh(A)
print("A = Q D Qᵀ:", np.allclose(Q @ np.diag(w) @ Q.T, A))

import numpy as np
 
A = np.array([[2.0, 1.0],
              [1.0, 2.0]])       # symmetric
 
vals, P = np.linalg.eig(A)
D = np.diag(vals)
 
# reconstruct A = P D P^-1
A_rebuilt = P @ D @ np.linalg.inv(P)
print("A = P D P⁻¹:", np.allclose(A, A_rebuilt))
 
# matrix power the cheap way: A^5 = P D^5 P^-1
A5_fast = P @ np.diag(vals**5) @ np.linalg.inv(P)
A5_slow = np.linalg.matrix_power(A, 5)
print("A⁵ matches:", np.allclose(A5_fast, A5_slow))
 
# symmetric ⇒ orthogonal P, so P^-1 = P^T
w, Q = np.linalg.eigh(A)
print("A = Q D Qᵀ:", np.allclose(Q @ np.diag(w) @ Q.T, A))

A = P D P⁻¹: True
A⁵ matches: True
A = Q D Qᵀ: True

A = P D P⁻¹: True
A⁵ matches: True
A = Q D Qᵀ: True

Why this matters for ML

• PCA is the eigendecomposition of the covariance matrix; eigenvalues rank directions by variance.
• Spectral methods (clustering, embeddings) read structure from the eigendecomposition of graph Laplacians.
• Matrix powers / dynamics: the largest eigenvalue governs the long-run behavior of iterated linear systems (Markov chains, recurrent dynamics, PageRank).

🧪 Try It Yourself

Exercise 1 – Reconstruct A from P and D

Exercise 2 – Fast matrix powers

Exercise 3 – Not everything diagonalizes

Recap

• Eigendecomposition $A = PDP^{-1}$: columns of $P$ are eigenvectors, diagonal of $D$ the eigenvalues — the matrix is diagonal in its eigenbasis.
• Exists iff $A$ has $n$ independent eigenvectors; symmetric matrices always qualify with orthogonal $P$ ($A = PDP^\top$).
• Makes powers ($A^k = PD^kP^{-1}$), determinants, and matrix functions trivial.
• Underlies PCA, spectral methods, and linear-dynamics analysis.

Next: the decomposition that works for any matrix, square or not — Singular Value Decomposition.