Why scale features?
Scaling is mainly important for ML algorithms that use distances or gradients:
- k-NN
- k-means
- SVM
- linear/logistic regression (often)
- neural networks
Tree-based models (like Random Forest) usually don’t need scaling.
Two common scalers
Min-Max scaling
Maps values to a fixed range (usually 0 to 1):
- x_scaled = (x - min) / (max - min)
Standardization (z-score scaling)
Centers and scales to mean=0 and std=1:
- x_scaled = (x - mean) / std
Example (using scikit-learn)
import numpy as np
from sklearn.preprocessing import MinMaxScaler, StandardScaler
x = np.array([[10], [20], [30], [100]])
mm = MinMaxScaler()
ss = StandardScaler()
print("MinMax:")
print(mm.fit_transform(x))
print("Standard:")
print(ss.fit_transform(x))Important rule: fit on train only
When doing ML:
- Fit scaler on training data
- Transform both train and test using that scaler
This avoids data leakage.
Visualize it
Here's how features move from raw, mismatched ranges to a common scale a model can use.
flowchart LR A["Raw features \(different ranges\)"] --> B["Fit scaler on training data"] B --> C["Transform features to a common scale"] C --> D["Feed to the model"] B -.->|"fit only on train"| E["Apply same scaler to test"] E --> D
Practical guidance
- If data has strong outliers: StandardScaler can be influenced; consider RobustScaler.
- If you need bounded values (0–1): use MinMaxScaler.
See the shift
Scaling doesn't change the shape of your data — it changes the ruler. The same four points below get remapped onto a 0–1 range, and again onto a mean-0, std-1 range.
🧪 Try It Yourself
Exercise 1 – Scale to [0, 1] with MinMaxScaler
Tab indents. Press Escape, then Tab, to leave the editor. Control plus Enter runs the code; Control plus Shift plus Enter submits it.
Press Run to see what your code prints.Exercise 2 – Standardize with StandardScaler
Tab indents. Press Escape, then Tab, to leave the editor. Control plus Enter runs the code; Control plus Shift plus Enter submits it.
Press Run to see what your code prints.Exercise 3 – Apply the manual z-score formula
Tab indents. Press Escape, then Tab, to leave the editor. Control plus Enter runs the code; Control plus Shift plus Enter submits it.
Press Run to see what your code prints.Check your understanding
What does Min-Max scaling compute?
Min-Max maps values to a fixed range, usually 0 to 1. The (x - mean) / std formula is standardization.
Show answer
A — (x - min) / (max - min) — Min-Max maps values to a fixed range, usually 0 to 1. The (x - mean) / std formula is standardization.
For x = 10, 20, 30 and 100, what does MinMaxScaler give the value 20?
(20 - 10) / (100 - 10) = 10 / 90, which is about 0.11. The value 100 maps to 1 and 10 maps to 0.
Show answer
A — About 0.11 — (20 - 10) / (100 - 10) = 10 / 90, which is about 0.11. The value 100 maps to 1 and 10 maps to 0.
To avoid data leakage, which approach is right for a scaler?
The scaler learns min, max, mean or std from the training data, and the same scaler transforms the test data.
Show answer
A — Fit on the training data only, then transform both train and test — The scaler learns min, max, mean or std from the training data, and the same scaler transforms the test data.
Which model type usually does not need feature scaling?
Distance or gradient based models need scaling. Trees split on thresholds, so scale matters little. With strong outliers consider RobustScaler.
Show answer
A — Tree-based models like Random Forest — Distance or gradient based models need scaling. Trees split on thresholds, so scale matters little. With strong outliers consider RobustScaler.
Next
With features on the same scale, turn categorical columns into numbers too — continue to One-Hot Encoding.