Feature Scaling (MinMax vs Standard)

Why scale features?

Scaling is mainly important for ML algorithms that use distances or gradients:

• k-NN
• k-means
• SVM
• linear/logistic regression (often)
• neural networks

Tree-based models (like Random Forest) usually don’t need scaling.

Two common scalers

Min-Max scaling

Maps values to a fixed range (usually 0 to 1):

• x_scaled = (x - min) / (max - min)

Standardization (z-score scaling)

Centers and scales to mean=0 and std=1:

• x_scaled = (x - mean) / std

Example (using scikit-learn)

MinMaxScaler vs StandardScaler

import numpy as np
from sklearn.preprocessing import MinMaxScaler, StandardScaler
 
x = np.array([[10], [20], [30], [100]])
 
mm = MinMaxScaler()
ss = StandardScaler()
 
print("MinMax:")
print(mm.fit_transform(x))
 
print("Standard:")
print(ss.fit_transform(x))

Important rule: fit on train only

When doing ML:

• Fit scaler on training data
• Transform both train and test using that scaler

This avoids data leakage.

Visualize it

Here’s how features move from raw, mismatched ranges to a common scale a model can use.

diagram Feature scaling pipeline mermaid

  flowchart LR
  A["Raw features \(different ranges\)"] --> B["Fit scaler on training data"]
  B --> C["Transform features to a common scale"]
  C --> D["Feed to the model"]
  B -.->|"fit only on train"| E["Apply same scaler to test"]
  E --> D

How raw features are scaled using a scaler fit on training data, then applied to test data.

Practical guidance

• If data has strong outliers: StandardScaler can be influenced; consider RobustScaler.
• If you need bounded values (0–1): use MinMaxScaler.

See the shift

Scaling doesn’t change the shape of your data — it changes the ruler. The same four points below get remapped onto a 0–1 range, and again onto a mean-0, std-1 range.

sketch The same values on three different scales p5.js Pause Restart

Raw values, min-max scaled values, and standardized values shown on their own number lines.

From the book

McKinney’s data.describe() already prints everything a scaler needs: mean, std, min, and the quartiles. StandardScaler is just (x - mean) / std using those two numbers; MinMaxScaler is just (x - min) / (max - min). Compute them once on the training set — the same way you’d run df.describe() once and reuse the summary — and apply the same numbers to test data.

🧪 Try It Yourself

Exercise 1 – Scale to [0, 1] with MinMaxScaler

pch.exerciseTag pch.exerciseTitle

Exercise 2 – Standardize with StandardScaler

pch.exerciseTag pch.exerciseTitle

Exercise 3 – Apply the manual z-score formula

pch.exerciseTag pch.exerciseTitle

Next

With features on the same scale, turn categorical columns into numbers too — continue to One-Hot Encoding.