We’ve met a zoo of matrix types across three chapters: square, invertible, symmetric, orthogonal, diagonal, positive definite, defective. This page is the family tree that organizes them all — a “phylogeny” showing which types are special cases of which, and which decomposition unlocks each. It’s the map you’ll return to whenever you meet a matrix and ask “what can I do with this?”

A real-life example: picking the right tool

You’re handed a matrix in a project. Is it invertible? Can I diagonalize it? Should I use Cholesky, eigendecomposition, or SVD? The phylogeny answers this by classification: check a few properties (square? symmetric? positive definite?) and the tree tells you exactly which fast, stable algorithm applies — instead of guessing or reaching for the most expensive general routine.

The family tree

Every arrow means “is a special case of.” The blue labels are the operations/decompositions each type unlocks.

  flowchart TD
  REAL["Real matrices ℝ^(n×m)
∃ pseudo-inverse, ∃ SVD"]
  REAL --> SQ["Square ℝ^(n×n)
∃ determinant, ∃ trace"]
  REAL --> NSQ["Non-square (n ≠ m)"]
  SQ -->|"det = 0"| SING["Singular (no inverse)"]
  SQ -->|"det ≠ 0"| INV["Invertible / Regular
∃ inverse"]
  SQ -->|"no eigenbasis"| DEF["Defective"]
  SQ -->|"basis of eigenvectors"| NDEF["Non-defective
(diagonalizable)"]
  NDEF -->|"AᵀA = AAᵀ"| NORM["Normal"]
  NORM --> SYM["Symmetric
eigenvalues ∈ ℝ"]
  SYM --> DIAG["Diagonal"]
  SYM --> PD["Positive definite
Cholesky, eigenvalues > 0"]
  DIAG --> ID["Identity matrix"]
  NORM -->|"AᵀA = AAᵀ = I"| ORTH["Orthogonal
columns orthonormal, A⁻¹ = Aᵀ"]
  ORTH --> ROTN["Rotation"]
  ORTH -.->|"subset of"| INV

(Adapted from Figure 4.13 of the book.)

The types, at a glance

Classify a matrix live

Watch a $2\times2$ matrix morph between an identity, a diagonal scaling, a symmetric matrix, a rotation, and a singular matrix. The badges light up as each property becomes true — this is the phylogeny check running in real time:

When the matrix collapses to the singular one, “invertible” goes dark and the square flattens to a line — det zero, exactly as the tree predicts.

Which decomposition should I use?

  flowchart TD
  Q["What is my matrix?"] --> NS["Non-square?"]
  NS -->|yes| SVD["SVD (only option)"]
  NS -->|no, square| SPD["Symmetric positive definite?"]
  SPD -->|yes| CH["Cholesky (fastest) — or eigendecomposition"]
  SPD -->|no| SYMq["Symmetric?"]
  SYMq -->|yes| EIGSYM["Eigendecomposition A = PDPᵀ (orthogonal P)"]
  SYMq -->|no| DIAGq["Diagonalizable?"]
  DIAGq -->|yes| EIG["Eigendecomposition A = PDP⁻¹"]
  DIAGq -->|no / unsure| SVD2["SVD — always works"]

NumPy: a property checker

import numpy as np
 
def classify(A):
    props = {}
    square = A.shape[0] == A.shape[1]
    props["square"] = square
    if square:
        det = np.linalg.det(A)
        props["invertible"] = not np.isclose(det, 0)
        props["symmetric"] = np.allclose(A, A.T)
        props["orthogonal"] = np.allclose(A.T @ A, np.eye(A.shape[0]))
        if props["symmetric"]:
            props["positive_definite"] = np.all(np.linalg.eigvalsh(A) > 0)
    return props
 
print("rotation:", classify(np.array([[0.0, -1.0], [1.0, 0.0]])))
print("SPD     :", classify(np.array([[2.0, 1.0], [1.0, 2.0]])))
print("singular:", classify(np.array([[1.0, 2.0], [2.0, 4.0]])))

import numpy as np
 
def classify(A):
    props = {}
    square = A.shape[0] == A.shape[1]
    props["square"] = square
    if square:
        det = np.linalg.det(A)
        props["invertible"] = not np.isclose(det, 0)
        props["symmetric"] = np.allclose(A, A.T)
        props["orthogonal"] = np.allclose(A.T @ A, np.eye(A.shape[0]))
        if props["symmetric"]:
            props["positive_definite"] = np.all(np.linalg.eigvalsh(A) > 0)
    return props
 
print("rotation:", classify(np.array([[0.0, -1.0], [1.0, 0.0]])))
print("SPD     :", classify(np.array([[2.0, 1.0], [1.0, 2.0]])))
print("singular:", classify(np.array([[1.0, 2.0], [2.0, 4.0]])))

rotation: {'square': True, 'invertible': True, 'symmetric': False, 'orthogonal': True}
SPD     : {'square': True, 'invertible': True, 'symmetric': True, 'orthogonal': False, 'positive_definite': True}
singular: {'square': True, 'invertible': False, 'symmetric': True, 'orthogonal': False, 'positive_definite': False}

rotation: {'square': True, 'invertible': True, 'symmetric': False, 'orthogonal': True}
SPD     : {'square': True, 'invertible': True, 'symmetric': True, 'orthogonal': False, 'positive_definite': True}
singular: {'square': True, 'invertible': False, 'symmetric': True, 'orthogonal': False, 'positive_definite': False}

Why this matters for ML

• Choosing the right solver: SPD covariance → Cholesky; symmetric → eigheigh; anything else → SVD. Picking correctly is faster and more numerically stable.
• Debugging: “singular matrix” and “not positive definite” errors are the tree telling you your matrix isn’t the type your algorithm assumed.
• Guarantees: knowing a matrix is symmetric PD (like a kernel or covariance) guarantees real eigenvalues, a Cholesky factor, and convex quadratic behavior.

🧪 Try It Yourself

Exercise 1 – Classify a rotation

Exercise 2 – Symmetric and positive definite?

Exercise 3 – Pick the decomposition

Recap

• Matrices form a hierarchy: real ⊃ square ⊃ (invertible, diagonalizable ⊃ normal ⊃ symmetric ⊃ positive-definite / diagonal / orthogonal).
• Each type unlocks specific tools: det/eigenvalues (square), Cholesky (SPD), $A^{-1}=A^\top$ (orthogonal), SVD (everything).
• Classifying a matrix tells you the fastest, most stable algorithm to use.

🎓 Chapter 4 complete

You’ve built the decomposition toolkit: determinant & trace → eigenvalues → Cholesky → eigendecomposition → SVD → low-rank approximation → the phylogeny. These factorizations are the literal engines of PCA, recommender systems, PageRank, Gaussian models, and compression.

Head back to the Matrix Decompositions Overview, or revisit Chapter 3 — Analytic Geometry.