Two ways models can be wrong

Bias (systematic error)

High bias means the model is too simple and misses real patterns.

• underfits
• training error is high
• validation error is high

Variance (sensitivity to noise)

High variance means the model is too complex and learns noise.

• overfits
• training error is low
• validation error is high

  flowchart LR
  B[High bias] --> U[Underfitting]
  V[High variance] --> O[Overfitting]

The tradeoff

As model complexity increases:

• bias tends to decrease
• variance tends to increase

Goal: find a sweet spot.

How to reduce bias

• add more features
• use a more flexible model
• reduce regularization

How to reduce variance

• collect more data
• increase regularization
• simplify the model
• use bagging/ensembles

Mini-checkpoint

If training and validation accuracy are both low:

• bias or variance?

(Usually bias / underfitting.)

🧪 Try It Yourself

Exercise 1 – Train-Test Split

Exercise 2 – Fit a Linear Model

Exercise 3 – Evaluate with MSE