A matrix is a rectangular grid of numbers. That sounds humble, but it is the single most important object in all of machine learning. A matrix can be data (a spreadsheet, an image, a batch of feature vectors) and it can be an action (a rotation, a projection, a neural network layer). Learning to see both faces of a matrix is the key skill of this page.

A real-life example: a grayscale image

Open any grayscale photo and zoom in far enough and you’ll find a grid of brightness values, each from 0 (black) to 255 (white). A 1080×1920 photo is literally a matrix with 1080 rows and 1920 columns. Every image filter you’ve ever used — blur, sharpen, edge-detect — is matrix arithmetic on that grid.

Definition

A real-valued $(m, n)$ matrix $A$ has $m$ rows and $n$ columns:

The entry $a_{ij}$ sits in row $i$, column $j$. A $(1, n)$ matrix is a row vector; an $(m, 1)$ matrix is a column vector. The set of all real $(m,n)$ matrices is written $\mathbb{R}^{m \times n}$.

Addition: element by element

Two matrices of the same shape add entry-by-entry:

That’s it — no surprises. Scaling is just as simple: $\lambda A$ multiplies every entry by $\lambda$.

Multiplication: the one that trips everyone up

Matrix multiplication is not element-wise. To multiply $A \in \mathbb{R}^{m\times n}$ by $B \in \mathbb{R}^{n \times k}$, each entry of the product $C = AB$ is a dot product of a row of $A$ with a column of $B$:

The inner dimensions must match — the number of columns of $A$ must equal the number of rows of $B$ — and the result has the outer dimensions:

  flowchart LR
  A["A
m × n"] --> C["C = AB
m × k"]
  B["B
n × k"] --> C
  N["inner dims n must match"] -.-> C

Worked example

For $A = \begin{bmatrix} 1 & 2 & 3\\ 3 & 2 & 1\end{bmatrix}$ and $B = \begin{bmatrix} 0 & 2\\ 1 & -1\\ 0 & 1\end{bmatrix}$:

Same two matrices — different shapes out. That alone proves $AB \neq BA$.

The identity, inverse, and transpose

The identity matrix $I_n$ has 1s on the diagonal and 0s elsewhere; it’s the “do nothing” matrix: $I_m A = A I_n = A$.

A square matrix $A$ has an inverse $A^{-1}$ if $AA^{-1} = A^{-1}A = I$. Not every matrix has one — those that do are called invertible / regular / nonsingular. For a $2\times 2$ matrix there’s a closed form:

valid exactly when $ad - bc \neq 0$. That quantity $ad - bc$ is the determinant; when it’s zero, the matrix collapses the plane and cannot be undone.

The transpose $A^\top$ flips rows and columns: $(A^\top)_{ij} = a_{ji}$. A matrix is symmetric if $A = A^\top$. Two identities worth memorizing:

Both reverse the order — a small fact that saves hours of debugging.

The second face: a matrix bends space

Here’s the idea that unlocks deep learning. Multiplying every point $\mathbf{x}$ of the plane by a fixed $2\times 2$ matrix $A$ transforms the whole plane — it rotates, scales, shears, or reflects it. Watch a unit grid (and the little house on it) get transformed as the matrix morphs between identity, a rotation, a shear, and a scaling:

The amber and violet arrows are the images of the basis vectors $\mathbf{e}_1 = (1,0)$ and $\mathbf{e}_2 = (0,1)$. A matrix is completely described by where it sends the basis vectors — those images are exactly its columns. Hold that thought; it’s the whole content of the Linear Mappings page.

NumPy: all of it in code

import numpy as np
 
A = np.array([[1, 2, 3],
              [3, 2, 1]])
B = np.array([[0, 2],
              [1, -1],
              [0, 1]])
 
print("A @ B =\n", A @ B)          # matrix product (2x2)
print("shape:", (A @ B).shape)
 
# transpose
print("A.T =\n", A.T)
 
# identity, inverse, determinant (square matrices)
M = np.array([[4.0, 7.0],
              [2.0, 6.0]])
print("det(M) =", round(np.linalg.det(M), 4))
print("inv(M) =\n", np.linalg.inv(M))
print("M @ inv(M) =\n", np.round(M @ np.linalg.inv(M), 6))   # ~ identity

import numpy as np
 
A = np.array([[1, 2, 3],
              [3, 2, 1]])
B = np.array([[0, 2],
              [1, -1],
              [0, 1]])
 
print("A @ B =\n", A @ B)          # matrix product (2x2)
print("shape:", (A @ B).shape)
 
# transpose
print("A.T =\n", A.T)
 
# identity, inverse, determinant (square matrices)
M = np.array([[4.0, 7.0],
              [2.0, 6.0]])
print("det(M) =", round(np.linalg.det(M), 4))
print("inv(M) =\n", np.linalg.inv(M))
print("M @ inv(M) =\n", np.round(M @ np.linalg.inv(M), 6))   # ~ identity

A @ B =
 [[2 3]
 [2 5]]
shape: (2, 2)
A.T =
 [[1 3]
 [2 2]
 [3 1]]
det(M) = 10.0
inv(M) =
 [[ 0.6 -0.7]
 [-0.2  0.4]]
M @ inv(M) =
 [[1. 0.]
 [0. 1.]]

A @ B =
 [[2 3]
 [2 5]]
shape: (2, 2)
A.T =
 [[1 3]
 [2 2]
 [3 1]]
det(M) = 10.0
inv(M) =
 [[ 0.6 -0.7]
 [-0.2  0.4]]
M @ inv(M) =
 [[1. 0.]
 [0. 1.]]

Why this matters for ML

• A fully-connected neural network layer is literally $\mathbf{y} = W\mathbf{x} + \mathbf{b}$ — a matrix multiply plus a vector. Stacking layers stacks matrix multiplies.
• Batching: instead of one vector at a time, we stack many inputs as columns of a matrix and transform them all with a single W @ XW @ X. GPUs exist to do this fast.
• The transpose shows up every time you backpropagate; the inverse and determinant decide whether a linear system (or a covariance matrix) is well-behaved.

🧪 Try It Yourself

Exercise 1 – Multiply two matrices

Exercise 2 – Show AB ≠ BA

Exercise 3 – Invert a 2×2 matrix

Recap

• A matrix is an $m\times n$ grid; it is both data and a transformation.
• Addition/scaling are element-wise; multiplication is rows-times-columns and needs matching inner dimensions.
• Multiplication is not commutative ($AB \neq BA$).
• The identity does nothing, the inverse undoes, the transpose flips — and $(AB)^\top = B^\top A^\top$, $(AB)^{-1} = B^{-1}A^{-1}$ both reverse order.

Next: the algorithm that actually solves $A\mathbf{x} = \mathbf{b}$ and finds inverses — Solving Systems of Linear Equations.