Underfitting

Underfitting happens when the model is too simple.

Symptoms:

• training score is poor
• validation score is also poor

Fixes:

• add features
• use a stronger model (e.g., trees/boosting)
• lower regularization

Overfitting

Overfitting happens when the model learns noise.

Symptoms:

• training score is great
• validation/test score is much worse

Fixes:

• simplify model
• add regularization
• get more data
• use cross-validation

Visual intuition

  flowchart TD
  A[Model complexity] --> B[Underfit (too simple)]
  A --> C[Good fit]
  A --> D[Overfit (too complex)]

Mini-checkpoint

Your model has:

• train accuracy: 99%
• validation accuracy: 75%

What is happening?

(Overfitting.)