What a cost function is

A cost function measures how bad predictions are.

Training usually means:

find parameters that minimize cost.

For regression, the most common is Mean Squared Error (MSE).

MSE definition

For N samples:

MSE = (1/N) * Σ (yi - ŷi)²MSE = (1/N) * Σ (yi - ŷi)²

Why square?

• penalizes large errors more
• differentiable (good for optimization)

Intuition

If you miss by:

• 1 unit → error contributes 1
• 10 units → error contributes 100

MSE pushes the model to avoid big mistakes.

MSE vs RMSE

• MSE: squared units (harder to interpret)
• RMSE: square root of MSE, back to original units

Why train on MSE instead of RMSE

Hands-On Machine Learning points out that the most natural performance measure for regression is RMSE, but training algorithms actually minimize MSE instead. The reason is purely practical: MSE is simpler to work with mathematically (no square root to differentiate), and the value of θθ that minimizes MSE is the same value that minimizes RMSE — square-rooting a function doesn’t move its minimum.

  flowchart LR
  X["Features X"] --> H["Model hθ(x)"]
  H --> P["Predictions ŷ"]
  Y["True targets y"] --> E["Errors (ŷ - y)"]
  P --> E
  E --> S["Squared errors (ŷ - y)²"]
  S --> M["Average → MSE(θ)"]

Formally, for a hypothesis hθhθ over a training set of mm instances:

MSE(θ) = (1/m) · Σ (hθ(x⁽ⁱ⁾) − y⁽ⁱ⁾)²MSE(θ) = (1/m) · Σ (hθ(x⁽ⁱ⁾) − y⁽ⁱ⁾)²

This is the cost function J(θ)J(θ) that the Normal Equation solves in closed form, and the same function Gradient Descent will minimize iteratively in the next lesson.

Why the cost surface is a bowl (and why that’s great news)

For Linear Regression, MSE(θ)MSE(θ) is a convex function — pick any two points on its surface and the straight line joining them never dips below the curve itself. Practically, that guarantees two things:

• There’s only one global minimum — no smaller local dips for an optimizer to get stuck in.
• The surface has no sudden cliffs; its slope changes smoothly everywhere.

Picture the surface as an actual bowl, with the height at each point being the cost for that combination of parameters. The Normal Equation jumps straight to the bottom of the bowl in one step; Gradient Descent (the next lesson) walks down the slope instead, and because the bowl is convex, it’s mathematically guaranteed to reach that same bottom eventually — it just may take a while, especially if the bowl is stretched into a long oval because the input features have very different scales.

MAE: a more robust alternative

MSE squares every error, so a single big outlier can dominate the entire cost — one badly mispredicted point with error 100 contributes as much as 10,000 points each off by 1. Mean Absolute Error (MAE) swaps the square for an absolute value:

MAE = (1/N) * Σ |yi - ŷi|MAE = (1/N) * Σ |yi - ŷi|

Because MAE grows linearly with the size of a mistake instead of quadratically, it’s far less sensitive to a handful of outliers. The trade-off: MAE has a sharp corner at zero error (its derivative jumps abruptly there), which makes some optimization algorithms converge less smoothly than they do with MSE’s gentle bowl.

import numpy as np
 
y_true = np.array([10, 12, 14, 16, 50])       # 50 is a big outlier
y_pred = np.array([11, 13, 13, 17, 20])       # model badly misses the outlier
 
mse = np.mean((y_true - y_pred) ** 2)
mae = np.mean(np.abs(y_true - y_pred))
 
print("MSE:", round(mse, 2))   # dominated by the outlier's squared error
print("MAE:", round(mae, 2))   # grows much more gently

import numpy as np
 
y_true = np.array([10, 12, 14, 16, 50])       # 50 is a big outlier
y_pred = np.array([11, 13, 13, 17, 20])       # model badly misses the outlier
 
mse = np.mean((y_true - y_pred) ** 2)
mae = np.mean(np.abs(y_true - y_pred))
 
print("MSE:", round(mse, 2))   # dominated by the outlier's squared error
print("MAE:", round(mae, 2))   # grows much more gently

Rule of thumb: default to MSE for training (friendlier derivatives, same minimum as RMSE). Reach for MAE — or a hybrid like Huber loss — when your data has outliers you don’t want to dominate the fit.

Code example

import numpy as np
 
y_true = np.array([3, 5, 2, 7])
y_pred = np.array([2.5, 5.2, 1.8, 7.9])
 
mse = np.mean((y_true - y_pred) ** 2)
rmse = np.sqrt(mse)
 
print("MSE:", mse)
print("RMSE:", rmse)

import numpy as np
 
y_true = np.array([3, 5, 2, 7])
y_pred = np.array([2.5, 5.2, 1.8, 7.9])
 
mse = np.mean((y_true - y_pred) ** 2)
rmse = np.sqrt(mse)
 
print("MSE:", mse)
print("RMSE:", rmse)

Visualize it

Each pink square’s area is one point’s squared error — literally (error)²(error)² drawn to scale. Watch the line improve step by step: as it fits the data better, the squares shrink and the MSE readout ticks down toward its minimum, then a fresh dataset rolls in and it happens again:

Mini-checkpoint

If your target is “price” in dollars:

• which is easier to explain to a business stakeholder: MSE or RMSE?

(Usually RMSE.)

🧪 Try It Yourself

Exercise 1 – Compute MSE by Hand

Exercise 2 – Convert MSE to RMSE

Exercise 3 – Use Scikit-Learn’s mean_squared_error

Next

Continue to Gradient Descent Explained — see how models minimize this same MSE cost function iteratively, one downhill step at a time.