So far our functions output a single number. But many map vectors to vectors — a neural network layer takes a vector in and puts a vector out. The derivative of such a function is the Jacobian: a matrix collecting every partial derivative. Its determinant tells you how much the function stretches volume — the key to change-of-variables in probability.

A real-life example: a robot arm

A robot arm’s tip position $(x, y)$ is a function of its joint angles $(\theta_1, \theta_2)$ — a map from $\mathbb{R}^2$ to $\mathbb{R}^2$. The Jacobian relates a small change in joint angles to the resulting movement of the tip. Engineers use it constantly: to convert desired tip velocities into joint velocities, and to detect “singular” arm configurations where the Jacobian loses rank and the arm locks up.

The Jacobian

For a vector-valued function $\mathbf{f} : \mathbb{R}^n \to \mathbb{R}^m$,

the derivative is the Jacobian $J$ — an $m \times n$ matrix whose $(i, j)$ entry is $\partial f_i / \partial x_j$:

Each row is the gradient of one output component. The gradient of a scalar function ($m = 1$) is just a Jacobian with one row. A tidy special case: for a linear map $\mathbf{f}(\mathbf{x}) = A\mathbf{x}$, the Jacobian is simply $A$ itself.

The Jacobian determinant magnifies volume

Here’s the geometric payoff. Apply $\mathbf{f}$ to a tiny square and it becomes a tiny parallelogram; the ratio of their areas is $|\det J|$. Watch a unit square get transformed — the Jacobian determinant is the factor by which its area grows or shrinks:

When $\det J = 0$ the parallelogram collapses to a line — the map crushes area, and (as for the robot arm) the configuration is singular.

The chain rule as matrix multiplication

For a composition $\mathbf{h} = \mathbf{g} \circ \mathbf{f}$, the Jacobian is the product of the Jacobians:

Dimensions chain like matrix multiplication ($m\times k$ times $k\times n$). This clean composition is exactly why deep networks — long chains of vector-valued functions — can be differentiated mechanically.

Change of variables

The Jacobian determinant is the “stretch factor” in the change-of-variables formula for integrals and probability densities. When you transform a random variable, its density gets rescaled by $|\det J|$. This shows up directly in normalizing flows, where the log-det of the Jacobian is the trainable term that keeps probabilities normalized.

  flowchart TD
  F["f : ℝⁿ → ℝᵐ"] --> J["Jacobian J (m×n)
Jᵢⱼ = ∂fᵢ/∂xⱼ"]
  J --> R["each row = gradient of one output"]
  J --> D["|det J| = volume magnification"]
  D -.-> CV["change of variables (probability)"]
  J -.-> BP["Jacobian products = chain rule / backprop"]

NumPy

import numpy as np
 
# f(x1, x2) = [x1² x2,  x1 + sin(x2)]   ->  J is 2x2
def f(v):
    x1, x2 = v
    return np.array([x1**2 * x2, x1 + np.sin(x2)])
 
def jacobian(f, v, h=1e-6):
    n = len(v); fx = f(v); m = len(fx)
    J = np.zeros((m, n))
    for j in range(n):                 # perturb each input, one at a time
        vp = v.copy(); vp[j] += h
        J[:, j] = (f(vp) - fx) / h
    return J
 
v = np.array([2.0, 0.0])
J = jacobian(f, v)
print("Jacobian at (2, 0):\n", np.round(J, 4))
print("|det J| (area magnification):", round(abs(np.linalg.det(J)), 4))
 
# For a linear map f(x) = A x, the Jacobian IS A
A = np.array([[1.0, 2.0], [3.0, 4.0]])
print("linear map Jacobian == A:", np.allclose(jacobian(lambda v: A @ v, v), A))

import numpy as np
 
# f(x1, x2) = [x1² x2,  x1 + sin(x2)]   ->  J is 2x2
def f(v):
    x1, x2 = v
    return np.array([x1**2 * x2, x1 + np.sin(x2)])
 
def jacobian(f, v, h=1e-6):
    n = len(v); fx = f(v); m = len(fx)
    J = np.zeros((m, n))
    for j in range(n):                 # perturb each input, one at a time
        vp = v.copy(); vp[j] += h
        J[:, j] = (f(vp) - fx) / h
    return J
 
v = np.array([2.0, 0.0])
J = jacobian(f, v)
print("Jacobian at (2, 0):\n", np.round(J, 4))
print("|det J| (area magnification):", round(abs(np.linalg.det(J)), 4))
 
# For a linear map f(x) = A x, the Jacobian IS A
A = np.array([[1.0, 2.0], [3.0, 4.0]])
print("linear map Jacobian == A:", np.allclose(jacobian(lambda v: A @ v, v), A))

Jacobian at (2, 0):
 [[0.     4.    ]
 [1.     1.    ]]
|det J| (area magnification): 4.0
linear map Jacobian == A: True

Jacobian at (2, 0):
 [[0.     4.    ]
 [1.     1.    ]]
|det J| (area magnification): 4.0
linear map Jacobian == A: True

Why this matters for ML

• Backpropagation multiplies layer Jacobians together — the vector-valued chain rule at scale.
• Normalizing flows rely on the log-determinant of the Jacobian to compute exact likelihoods.
• Sensitivity analysis uses the Jacobian to see how outputs respond to input perturbations (adversarial robustness, feature attribution).

🧪 Try It Yourself

Exercise 1 – Build a Jacobian

Exercise 2 – Area magnification

Exercise 3 – Chain rule via Jacobian product

Recap

• The Jacobian is the $m\times n$ matrix of partial derivatives of a vector-valued function; each row is one output’s gradient.
• For a linear map $\mathbf{f}(\mathbf{x}) = A\mathbf{x}$, the Jacobian is $A$.
• $|\det J|$ is the volume magnification factor — the heart of change-of-variables.
• Jacobians multiply under composition — the vector chain rule behind backpropagation.

Next: derivatives with respect to matrices, which become tensors — Gradients of Matrices.