The goal

Regression learns a function:

• inputs (features) XX
• output (target) yy (continuous number)

So the model can predict ŷŷ for new inputs.

  flowchart LR
  X[Features] --> M[Regression Model] --> Y[Predicted target ŷ]

A toy example

Predict house price from size:

• X = size_sqftX = size_sqft
• y = pricey = price

Regression tries to find a “best-fit” relationship.

The general Linear Regression model

Géron’s Hands-On Machine Learning opens Chapter 4 with the most general form of a linear model: a weighted sum of the input features, plus a bias term:

ŷ = θ0 + θ1x1 + θ2x2 + ... + θnxnŷ = θ0 + θ1x1 + θ2x2 + ... + θnxn

• ŷŷ is the predicted value
• nn is the number of features
• xixi is the i-th feature value
• θjθj is the j-th model parameter, including the bias term θ0θ0 and the feature weights θ1θ1 to θnθn

More concisely, this is written in vectorized form:

ŷ = hθ(x) = θ · xŷ = hθ(x) = θ · x

where θθ is the parameter vector and xx is the feature vector (with x0 = 1x0 = 1 so the bias term slots neatly into the dot product). hθhθ is called the hypothesis function — every model in this phase (simple, multiple, polynomial, Ridge, Lasso) is just a different way of choosing θθ for this same shape of equation.

  flowchart LR
  X["Feature vector x
(x0=1, x1, x2, ...)"] -->|dot product with θ| H["Hypothesis hθ(x)"]
  H --> Y["Prediction ŷ"]

To train the model, we need a way to measure how wrong it is. The book uses Mean Squared Error (MSE) rather than Root Mean Squared Error (RMSE) for one practical reason: MSE is simpler to minimize mathematically, and whatever θθ minimizes MSE also minimizes RMSE (square-rooting doesn’t change where the minimum sits). You’ll meet MSE properly two lessons from now.

Two ways to train this model

Once you have a shape for hθ(x)hθ(x), “training” just means searching for the θθ that makes predictions land as close as possible to the real yy values. Hands-On Machine Learning frames the rest of this phase around two completely different routes to that same destination:

• A closed-form equation — plug the training data into a formula and get θθ back directly, in one shot. This is the Normal Equation, which you’ll use by hand in the very next lessons.
• An iterative search — start with a random guess for θθ and nudge it a little closer to the answer, step after step, until it settles near the minimum of the cost function. This is Gradient Descent.

For plain Linear Regression, both routes land on (almost) the same θθ — the difference is speed, and how well each approach scales as the number of features or training instances grows.

  flowchart LR
  T["Training = find θ
that minimizes a cost function"] --> N["Normal Equation
(closed-form, one shot)"]
  T --> G["Gradient Descent
(iterative, step by step)"]
  N --> S["(Almost) the same θ"]
  G --> S

Common regression use-cases

• forecasting (demand, sales)
• risk modeling (credit risk)
• resource planning
• personalization (expected spend)

Assumptions (important)

Different regression models make different assumptions.

Linear regression assumes (roughly):

• a linear relationship between features and target
• errors are random (noise)

These assumptions are often “wrong” but still useful.

Baselines matter

Before complex models, always try:

• predicting the mean
• simple linear regression

If your fancy model doesn’t beat a baseline, something’s off.

Visualize it

The hypothesis function hθ(x) = θ0 + θ1x1 + θ2x2 + ... hθ(x) = θ0 + θ1x1 + θ2x2 + ... is just a running total: start from the bias, then add one weighted feature at a time. Watch the bars build up term by term for a random house, then settle into the final prediction ŷŷ — then a new instance rolls in and it happens again:

Mini-checkpoint

Given a dataset, write down:

• your target column (y)
• 5 features (X)
• what a “good enough” error means for the business

🧪 Try It Yourself

Exercise 1 – Build the Feature Vector (add x0 = 1)

Exercise 2 – Predict with the Hypothesis Function

Exercise 3 – Baseline: Predict the Mean

Next

Continue to Simple Linear Regression — put the hypothesis function to work with just one feature and see what the weight and bias actually mean.