Chapter 3 Formula Sheet

Reference, not teaching. Each entry names the book’s equation or definition number, states the result, and says when to reach for it. Read the concept pages first; come back here when you are working.

The one-line summary

Add an inner product to a vector space and everything else follows:

$$\langle\mathbf{x},\mathbf{y}\rangle \;\Longrightarrow\; \lVert\mathbf{x}\rVert = \sqrt{\langle\mathbf{x},\mathbf{x}\rangle} \;\Longrightarrow\; d(\mathbf{x},\mathbf{y}) = \lVert\mathbf{x}-\mathbf{y}\rVert, \qquad \cos\omega = \frac{\langle\mathbf{x},\mathbf{y}\rangle}{\lVert\mathbf{x}\rVert\lVert\mathbf{y}\rVert}$$

§3.1 Norms

result	statement	when
Def 3.1 norm axioms	absolutely homogeneous $\lVert\lambda\mathbf{x}\rVert = \lvert\lambda\rvert\lVert\mathbf{x}\rVert$; triangle $\lVert\mathbf{x}+\mathbf{y}\rVert \leq \lVert\mathbf{x}\rVert + \lVert\mathbf{y}\rVert$; positive definite	checking a candidate “length”
Eq 3.3 Manhattan	$\lVert\mathbf{x}\rVert_1 = \sum_i \lvert x_i\rvert$	Lasso, robust loss, sparse recovery
Eq 3.4 Euclidean	$\lVert\mathbf{x}\rVert_2 = \sqrt{\mathbf{x}^\top\mathbf{x}}$	the book’s default everywhere after §3.1
maximum	$\lVert\mathbf{x}\rVert_\infty = \max_i\lvert x_i\rvert$	adversarial budgets, worst-case bounds
$p$-norm	$\lVert\mathbf{x}\rVert_p = \left(\sum_i\lvert x_i\rvert^p\right)^{1/p}$, $p \geq 1$	theory; rarely used directly
ordering	$\lVert\mathbf{x}\rVert_\infty \leq \lVert\mathbf{x}\rVert_q \leq \lVert\mathbf{x}\rVert_p \leq \lVert\mathbf{x}\rVert_1$ for $1 \leq p \leq q$	sanity-checking a computation
why $p \geq 1$	$\lVert\mathbf{e}_1+\mathbf{e}_2\rVert_p = 2^{1/p} > 2$ when $p < 1$	the triangle inequality fails; ball not convex

python

np.linalg.norm(x, 1)        # l1
np.linalg.norm(x)           # l2 for a vector, FROBENIUS for a matrix
np.linalg.norm(x, np.inf)   # maximum
np.linalg.norm(A, 2)        # matrix 2-norm: largest singular value

§3.2 Inner products

result	statement	when
bilinear	linear in each argument separately (Eq 3.6, 3.7)	automatic for $\mathbf{x}^\top\mathbf{A}\mathbf{y}$
Def 3.2 symmetric	$\Omega(\mathbf{x},\mathbf{y}) = \Omega(\mathbf{y},\mathbf{x})$ $\iff$ $\mathbf{A} = \mathbf{A}^\top$	the condition Exercise 3.2 breaks
Def 3.2 positive definite	$\mathbf{x}^\top\mathbf{A}\mathbf{x} > 0$ for $\mathbf{x} \neq \mathbf{0}$ (Eq 3.8)	makes the induced norm real
Def 3.3 inner product	bilinear + symmetric + positive definite	the whole definition
Eq 3.10 matrix form	$\langle\mathbf{x},\mathbf{y}\rangle = \hat{\mathbf{x}}^\top\mathbf{A}\hat{\mathbf{y}}$ with $A_{ij} = \langle\mathbf{b}_i,\mathbf{b}_j\rangle$	computing anything
Theorem 3.5	inner product $\iff$ symmetric positive definite $\mathbf{A}$ exists (Eq 3.15)	there are no other kinds
SPD consequences	trivial null space; every diagonal entry positive	necessary, not sufficient
the test	Cholesky succeeds $\iff$ positive definite	not eigvalsh(A) >= 0, which accepts semidefinite

python

np.allclose(A, A.T)         # symmetry
np.linalg.cholesky(A)       # raises LinAlgError iff not positive definite
np.linalg.eigvalsh(A)       # eigenvalues, if you want to see them

§3.3 Lengths and distances

result	statement	when
Eq 3.16 induced norm	$\lVert\mathbf{x}\rVert := \sqrt{\langle\mathbf{x},\mathbf{x}\rangle}$	every inner product gives a norm
Eq 3.17 Cauchy-Schwarz	$\lvert\langle\mathbf{x},\mathbf{y}\rangle\rvert \leq \lVert\mathbf{x}\rVert\lVert\mathbf{y}\rVert$, equality iff parallel	makes the cosine well defined; gives the triangle inequality
Def 3.6 distance	$d(\mathbf{x},\mathbf{y}) := \lVert\mathbf{x}-\mathbf{y}\rVert$ (Eq 3.21)	the length of a difference, nothing more
metric axioms	positive definite, symmetric (Eq 3.22), triangle (Eq 3.23)	a metric needs no vector space
the hierarchy	inner products $\subset$ norms $\subset$ metrics	which structure you actually have
parallelogram law	$\lVert\mathbf{x}+\mathbf{y}\rVert^2 + \lVert\mathbf{x}-\mathbf{y}\rVert^2 = 2\lVert\mathbf{x}\rVert^2 + 2\lVert\mathbf{y}\rVert^2$	test whether a norm comes from an inner product
Remark	inner product and metric run in opposite directions	similar = large inner product, small distance
expansion	$\lVert\mathbf{x}-\mathbf{y}\rVert^2 = \lVert\mathbf{x}\rVert^2 - 2\langle\mathbf{x},\mathbf{y}\rangle + \lVert\mathbf{y}\rVert^2$	why dot-product ranking $\neq$ distance ranking
Mahalanobis	$\sqrt{(\mathbf{x}-\mathbf{y})^\top\Sigma^{-1}(\mathbf{x}-\mathbf{y})}$	distances that respect correlation

§3.4 Angles and orthogonality

result	statement	when
Eq 3.24, 3.25	$\cos\omega = \dfrac{\langle\mathbf{x},\mathbf{y}\rangle}{\lVert\mathbf{x}\rVert\lVert\mathbf{y}\rVert}$, $\omega \in [0,\pi]$	unsigned angle; always clip before arccos
cosine similarity	the same ratio	comparing embeddings; scale-free
Def 3.7 orthogonal	$\langle\mathbf{x},\mathbf{y}\rangle = 0$, written $\mathbf{x}\perp\mathbf{y}$	$\mathbf{0}$ is orthogonal to everything
Def 3.7 orthonormal	orthogonal and both of unit length	the extra half §3.5 needs
Def 3.8 orthogonal matrix	$\mathbf{A}\mathbf{A}^\top = \mathbf{I} = \mathbf{A}^\top\mathbf{A}$, so $\mathbf{A}^{-1} = \mathbf{A}^\top$ (Eq 3.29, 3.30)	free inverse, no conditioning cost
Eq 3.31	$\lVert\mathbf{A}\mathbf{x}\rVert = \lVert\mathbf{x}\rVert$	lengths preserved
Eq 3.32	angles preserved	both from $\mathbf{A}^\top\mathbf{A} = \mathbf{I}$
concentration	$\mathrm{sd}(\cos\omega) \approx 1/\sqrt{d}$ for random pairs	why cosine works in high dimensions

Orthogonality is relative to the inner product. Example 3.7: $(1,1)$ and $(-1,1)$ are at $90°$ under the dot product and at $109.4712°$ under $\mathrm{diag}(2,1)$, with $\cos\omega = -\tfrac13$.

§3.5 Orthonormal basis

result	statement	when
Def 3.9 ONB	$\langle\mathbf{b}_i,\mathbf{b}_j\rangle = 0$ for $i \neq j$ (Eq 3.33), $\langle\mathbf{b}_i,\mathbf{b}_i\rangle = 1$ (Eq 3.34)	orthogonal alone gives only Eq 3.33
Gram matrix	$\mathbf{B}^\top\mathbf{B} = \mathbf{I}$	both conditions in one equation
coordinates	$\boldsymbol{\lambda} = \mathbf{B}^\top\mathbf{x}$, i.e. $\lambda_i = \langle\mathbf{b}_i,\mathbf{x}\rangle$	$O(n^2)$, no solve, independent per coordinate
Parseval	$\lVert\mathbf{x}\rVert^2 = \sum_i \lambda_i^2$	fails off an ONB; the basis of “variance explained”
orthogonal but not normal	$\lambda_i = \langle\mathbf{b}_i,\mathbf{x}\rangle / \langle\mathbf{b}_i,\mathbf{b}_i\rangle$	the division Fourier coefficients carry
numerics	classical Gram-Schmidt loses orthogonality like $\varepsilon\kappa^2$	use np.linalg.qr (Householder)

At $\kappa = 10^{10}$: classical Gram-Schmidt gives $\lVert\mathbf{Q}^\top\mathbf{Q}-\mathbf{I}\rVert = 1.013$ (orthogonality entirely gone), modified $2.507\times10^{-7}$, Householder $5.774\times10^{-16}$.

§3.6 Orthogonal complement

result	statement	when
definition	$U^\perp = \{\mathbf{v} : \langle\mathbf{v},\mathbf{u}\rangle = 0 \ \forall\,\mathbf{u}\in U\}$	it is a subspace
dimension	$\dim U + \dim U^\perp = D$	the accounting that makes truncation exact
intersection	$U \cap U^\perp = \{\mathbf{0}\}$	from positive definiteness
checking	orthogonal to a basis of $U$ suffices	$U^\perp$ = null space of $U$‘s basis as rows
Eq 3.36	$\mathbf{x} = \sum_m \lambda_m\mathbf{b}_m + \sum_j \psi_j\mathbf{b}_j^\perp$, uniquely	the split PCA relies on
Pythagoras	$\lVert\mathbf{x}\rVert^2 = \lVert\mathbf{p}\rVert^2 + \lVert\mathbf{q}\rVert^2$	because $\langle\mathbf{p},\mathbf{q}\rangle = 0$
normal vector	$\dim U^\perp = 1$: one vector $\mathbf{w}$ describes all of $U$	hyperplanes, Ch 12
point-to-plane	$d(\mathbf{x},U) = \dfrac{\lvert\langle\mathbf{w},\mathbf{x}\rangle\rvert}{\lVert\mathbf{w}\rVert}$	SVM margin
residual operator	$\mathbf{I} - \mathbf{P}$; rank $= \dim U^\perp$	keeps what $\mathbf{P}$ discards

§3.7 Inner product of functions

result	statement	when
Eq 3.37	$\langle u,v\rangle = \displaystyle\int_a^b u(x)v(x)\,dx$	the interval is part of the definition
norm	$\lVert u\rVert = \sqrt{\int_a^b u(x)^2\,dx}$	$L^2$ norm
Example 3.9	$\langle\sin,\cos\rangle = 0$ on $[-\pi,\pi]$	the product is odd; exact cancellation
Eq 3.38	$\{1, \cos x, \cos 2x, \dots, \sin x, \sin 2x, \dots\}$ pairwise orthogonal	the Fourier basis
squared norms	$\lVert 1\rVert^2 = 2\pi$, $\lVert\cos kx\rVert^2 = \lVert\sin kx\rVert^2 = \pi$	orthogonal, not orthonormal
coefficient	$c_k = \dfrac{1}{\pi}\displaystyle\int_{-\pi}^{\pi} f(x)\sin(kx)\,dx$	the $1/\pi$ is $1/\lVert\sin kx\rVert^2$
truncation error	total energy minus the energy kept	Parseval, predictive
caveat	$L^2$ convergence $\neq$ pointwise; Gibbs overshoot $\to \approx 1.179$	reconstructing edges

§3.8 Orthogonal projections

The section to memorise.

result	statement	when
Def 3.10	$\pi^2 = \pi$, i.e. $\mathbf{P}_\pi^2 = \mathbf{P}_\pi$	idempotence alone permits oblique projections
orthogonal projection	idempotent and symmetric	the closest point
Eq 3.40 line, coordinate	$\lambda = \dfrac{\mathbf{b}^\top\mathbf{x}}{\mathbf{b}^\top\mathbf{b}}$	one direction
Eq 3.42 line, projection	$\pi_U(\mathbf{x}) = \lambda\mathbf{b}$	—
Eq 3.44	$\lVert\pi_U(\mathbf{x})\rVert = \lvert\cos\omega\rvert\,\lVert\mathbf{x}\rVert$	geometric reading
Eq 3.46 line, matrix	$\mathbf{P}_\pi = \dfrac{\mathbf{b}\mathbf{b}^\top}{\mathbf{b}^\top\mathbf{b}}$	symmetric, rank 1
Eq 3.56 normal equation	$\mathbf{B}^\top\mathbf{B}\boldsymbol{\lambda} = \mathbf{B}^\top\mathbf{x}$	the whole of §3.8.2
Eq 3.57 pseudo-inverse	$\boldsymbol{\lambda} = (\mathbf{B}^\top\mathbf{B})^{-1}\mathbf{B}^\top\mathbf{x}$	needs $\mathbf{B}$ of full column rank
Eq 3.58 projection	$\pi_U(\mathbf{x}) = \mathbf{B}(\mathbf{B}^\top\mathbf{B})^{-1}\mathbf{B}^\top\mathbf{x}$	—
Eq 3.59 matrix	$\mathbf{P}_\pi = \mathbf{B}(\mathbf{B}^\top\mathbf{B})^{-1}\mathbf{B}^\top$	derivations, not computation
Eq 3.63 projection error	$\lVert\mathbf{x} - \pi_U(\mathbf{x})\rVert$	the objective PCA minimises
Eq 3.65, 3.66 ONB case	$\pi_U(\mathbf{x}) = \mathbf{B}\mathbf{B}^\top\mathbf{x}$, $\boldsymbol{\lambda} = \mathbf{B}^\top\mathbf{x}$	always prefer this
spectrum	eigenvalues $\in \{0,1\}$; $\mathrm{rank} = \mathrm{trace} = \dim U$	fastest diagnostic
Eq 3.67, 3.68 Gram-Schmidt	$\mathbf{u}_1 := \mathbf{b}_1$; $\mathbf{u}_k := \mathbf{b}_k - \pi_{\mathrm{span}[\mathbf{u}_1..\mathbf{u}_{k-1}]}(\mathbf{b}_k)$	building an ONB
Eq 3.72 affine	$\pi_L(\mathbf{x}) = \mathbf{x}_0 + \pi_U(\mathbf{x}-\mathbf{x}_0)$	shift, project, shift back
Eq 3.73 affine distance	$d(\mathbf{x},L) = d(\mathbf{x}-\mathbf{x}_0,\ \pi_U(\mathbf{x}-\mathbf{x}_0))$	translation preserves distance
ridge / jitter	$\mathbf{B}^\top\mathbf{B} + \varepsilon\mathbf{I}$	stabilises the solve; no longer a projection
general inner product	$\mathbf{B}^\top\mathbf{A}\mathbf{B}\boldsymbol{\lambda} = \mathbf{B}^\top\mathbf{A}\mathbf{x}$	Exercise 3.6; $\mathbf{P}$ is then not symmetric

python

lam, *_ = np.linalg.lstsq(B, x, rcond=None)   # the right default: tolerates rank deficiency
proj = B @ lam
Q, _ = np.linalg.qr(B); proj = Q @ (Q.T @ x)  # same answer, explicit ONB
P = B @ np.linalg.inv(B.T @ B) @ B.T          # for a derivation, not production

Never form $\mathbf{B}^\top\mathbf{B}$ in production: $\kappa(\mathbf{B}^\top\mathbf{B}) = \kappa(\mathbf{B})^2$. On a degree-11 monomial design ($\kappa(\mathbf{B}) = 1.2\times10^{8}$) the normal-equation projection is $0.048$ away from the QR answer.

§3.9 Rotations

result	statement	when
rotation	orthogonal and $\det = +1$	orientation preserved; $\det = -1$ is a reflection
Eq 3.76 $\mathbb{R}^2$	$\mathbf{R}(\theta) = \begin{bmatrix}\cos\theta & -\sin\theta\\ \sin\theta & \cos\theta\end{bmatrix}$	columns are the images of $\mathbf{e}_1, \mathbf{e}_2$
composition in $\mathbb{R}^2$	$\mathbf{R}(\alpha)\mathbf{R}(\beta) = \mathbf{R}(\alpha+\beta)$	Abelian group
Eq 3.77 about $\mathbf{e}_1$	$\begin{bmatrix}1&0&0\\0&\cos\theta&-\sin\theta\\0&\sin\theta&\cos\theta\end{bmatrix}$	fixes $x_1$
Eq 3.78 about $\mathbf{e}_2$	$\begin{bmatrix}\cos\theta&0&\sin\theta\\0&1&0\\-\sin\theta&0&\cos\theta\end{bmatrix}$	signs reversed — not a typo
Eq 3.79 about $\mathbf{e}_3$	$\begin{bmatrix}\cos\theta&-\sin\theta&0\\\sin\theta&\cos\theta&0\\0&0&1\end{bmatrix}$	the 2-D case, padded
Def 3.11 Givens	the identity with $r_{ii}=\cos\theta$, $r_{ij}=-\sin\theta$, $r_{ji}=\sin\theta$, $r_{jj}=\cos\theta$ (Eq 3.80, 3.81)	rotate one plane, fix $n-2$ dimensions
parameter count	$n(n-1)/2$ angles	1, 3, 6, 45 for $n = 2,3,4,10$
§3.9.4 properties	preserve distances; preserve angles; not commutative in $\mathbb{R}^{\geq 3}$; group only about a shared fixed point	Euler-angle ordering matters

Measured: $\lVert\mathbf{R}_1(35°)\mathbf{R}_3(70°) - \mathbf{R}_3(70°)\mathbf{R}_1(35°)\rVert_F = 0.961059$ in space, against $1.570\times10^{-16}$ in the plane.

The determinant as a one-glance diagnostic

$\det\mathbf{A}$	with $\mathbf{A}^\top\mathbf{A} = \mathbf{I}$	what it is
$+1$	yes	rotation
$-1$	yes	reflection
$0$	no	singular; a projection if also idempotent
$c^n$	no	uniform scaling by $c$
$1$	no	area-preserving but distorting — a shear

Numbers worth remembering

quantity	value	from
$\lVert(1.6, 0.9)\rVert_1, \lVert\cdot\rVert_2, \lVert\cdot\rVert_\infty$	$2.5$, $1.835756$, $1.6$	Norms
$\lVert\mathbf{e}_1+\mathbf{e}_2\rVert_{0.6}$ vs $2$	$3.174802$ vs $2$ — fails	Norms
eigenvalues of $\begin{bmatrix}9&6\\6&5\end{bmatrix}$, $\begin{bmatrix}9&6\\6&3\end{bmatrix}$	$(0.675, 13.325)$, $(-0.708, 12.708)$	Example 3.4
boundary of $\begin{bmatrix}9&6\\6&a\end{bmatrix}$	$a = 4$ exactly	Inner Products
$\lVert(1,1)\rVert$ under dot, Eq 3.19	$\sqrt2 = 1.414214$, exactly $1$	Example 3.5
Example 3.6 angle	$\cos = 3/\sqrt{10}$, $\omega = 18.434949°$	§3.4
Example 3.7 angle	$90°$ then $109.4712°$, $\cos = -1/3$	§3.4
$\mathrm{sd}(\cos\omega)$ at $d = 10^4$	$0.00981 \approx 1/\sqrt{d}$	Angles
classical Gram-Schmidt at $\kappa = 10^{10}$	$\lVert\mathbf{Q}^\top\mathbf{Q}-\mathbf{I}\rVert = 1.013$	ONB
digits data	rank $61$ of $64$; $21$ components for $90\%$	Complement, Projections
digits complement	span of pixels $0$, $32$, $39$	Complement
square wave $c_1$, $c_3$	$4/\pi = 1.273240$, $4/(3\pi) = 0.424413$	Functions
Gibbs overshoot	$\to 1.179$, not $1$	Functions
Example 3.10	$\mathbf{P} = \tfrac19\begin{bmatrix}1&2&2\\2&4&4\\2&4&4\end{bmatrix}$, $\pi(\mathbf{x}) = \tfrac19(5,10,10)$	§3.8.1
Example 3.11	$\boldsymbol{\lambda} = (5,-3)$, $\pi = (5,2,-1)$, error $\sqrt6$	§3.8.2
Example 3.11 matrix	$\mathbf{P} = \tfrac16\begin{bmatrix}5&2&-1\\2&2&2\\-1&2&5\end{bmatrix}$	§3.8.2
Exercise 3.5	$\pi = (1,-5,-1,-2,3)$, $d = \sqrt{60}$, rank $3$ not $4$	Exercises
Exercise 3.6	$\pi_U(\mathbf{e}_2) = (0.5, 0, -0.5)$, $d = 1$ exactly	Exercises
Book Eq 3.74	$\det = 0.9908$ — the printed matrix is not orthogonal	Rotations
3-D commutator at $35°, 70°$	$0.961059$	Rotations

Where each section is used later

from Chapter 3	used in
norms (§3.1)	Ch 7 regularised objectives; Ch 9 ridge and Lasso
inner products (§3.2)	Ch 12 kernels; Ch 6 covariance
distances (§3.3)	Ch 11 clustering; nearest-neighbour anything
angles (§3.4)	embedding comparison; whitening
ONB (§3.5)	Ch 4 — eigendecomposition and SVD produce them
complement (§3.6)	Ch 10 discarded directions; Ch 9 residual space
function inner products (§3.7)	Ch 6 — the analogous move for random variables
projections (§3.8)	Ch 9 least squares; Ch 10 PCA
affine projections (§3.8.4)	Ch 12 separating hyperplane
rotations (§3.9)	Ch 4 — the outer factors of an SVD

The five mistakes this chapter is designed to prevent

1. Assuming the dot product. Orthogonality, length, angle and projection all depend on the inner product. Exercise 3.6 gives three different wrong answers from three different versions of this assumption.
2. Testing definiteness and calling it an inner product. Symmetry is a separate condition, and Exercise 3.2’s matrix passes the definiteness test.
3. Testing eigvalsh(A) >= 0. That accepts positive semidefinite matrices, where a nonzero vector has length zero. Use Cholesky.
4. Applying Equation 3.59 to a spanning set. It needs a basis. Exercise 3.5’s four vectors have rank 3, and the Gram matrix’s condition number is $2\times10^{17}$.
5. Forming $\mathbf{B}^\top\mathbf{B}$ in production code. It squares the condition number. Use lstsq or QR.

pch.quizTag Sheet check

1. You need to project onto a subspace given by a matrix B whose columns may be linearly dependent. Which call?

   A B @ np.linalg.inv(B.T @ B) @ B.T @ x B np.linalg.lstsq(B, x, rcond=None), because it uses a minimum-norm pseudo-inverse and does not require full column rank C np.linalg.solve(B, x) D np.linalg.cholesky(B.T @ B)
   pch.quizShowAnswer

   B — np.linalg.lstsq(B, x, rcond=None), because it uses a minimum-norm pseudo-inverse and does not require full column rank — Equation 3.59 requires a basis, and a rank-deficient B makes B-transpose B singular. lstsq returns the correct projection regardless, and it also avoids squaring the condition number.

2. Which single number tells you the dimension of the subspace a projection matrix projects onto?

   A Its largest eigenvalue B Its trace, which equals its rank because the eigenvalues are only 0 and 1 C Its determinant D Its Frobenius norm
   pch.quizShowAnswer

   B — Its trace, which equals its rank because the eigenvalues are only 0 and 1 — P squared equals P forces lambda squared equals lambda for every eigenvalue, so each is 0 or 1. The trace is their sum and the rank is the count of nonzeros, so both equal the dimension.

3. A colleague reports that a norm they wrote satisfies the parallelogram law. What does that tell you?

   A Nothing in particular B That it comes from an inner product, so it has angles, orthogonality and projections attached — which l1 and l-infinity do not C That it is the l1 norm D That it is not a norm
   pch.quizShowAnswer

   B — That it comes from an inner product, so it has angles, orthogonality and projections attached — which l1 and l-infinity do not — The parallelogram law is exactly the test. Among the standard p-norms only p = 2 passes, which is why ridge has a closed-form solution and Lasso does not.

Recall card

• One inner product generates the whole chapter: length is its square root on the diagonal, distance is the length of a difference, and the cosine is it divided by two lengths.
• Theorem 3.5 is an if and only if — every inner product on a finite-dimensional real space is a symmetric positive definite matrix, and testing a candidate means testing that matrix.
• Cholesky is the definiteness test, because comparing eigenvalues to zero accepts the semidefinite boundary case.
• The projection recipe is three steps: solve the normal equation for the coordinates, multiply by the basis for the projection, and sandwich the inverse Gram matrix for the projection matrix — and all three collapse when the basis is orthonormal.
• A projection matrix has eigenvalues zero and one only, so its rank and trace both equal the dimension it projects onto.
• A rotation is an orthogonal matrix of determinant plus one, needs n(n−1)/2 angles in n dimensions, and does not commute beyond the plane.
• Never form B-transpose B in production: it squares the condition number, which is the same failure behind classical Gram-Schmidt.

Back to: Analytic Geometry Overview — or forward to Chapter 4, where the orthonormal bases of §3.5 turn out to be what every decomposition produces.