Why pipelines?
Pipelines help you:
- Avoid data leakage
- Keep preprocessing consistent across train/test
- Make training reproducible
Example: numeric + categorical preprocessing
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import OneHotEncoder, StandardScaler
from sklearn.linear_model import LogisticRegression
# Example dataset
df = pd.DataFrame({
"age": [22, 25, None, 31, 35],
"city": ["Pune", "Delhi", "Pune", "Mumbai", None],
"score": [60, 65, 70, 75, 80],
"target": [0, 0, 1, 1, 1],
})
X = df.drop(columns=["target"])
y = df["target"]
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, random_state=42, stratify=y
)
numeric_features = ["age", "score"]
cat_features = ["city"]
numeric_transformer = Pipeline(steps=[
("imputer", SimpleImputer(strategy="median")),
("scaler", StandardScaler()),
])
categorical_transformer = Pipeline(steps=[
("imputer", SimpleImputer(strategy="most_frequent")),
("onehot", OneHotEncoder(handle_unknown="ignore")),
])
preprocess = ColumnTransformer(
transformers=[
("num", numeric_transformer, numeric_features),
("cat", categorical_transformer, cat_features),
]
)
model = Pipeline(steps=[
("preprocess", preprocess),
("clf", LogisticRegression(max_iter=1000)),
])
model.fit(X_train, y_train)
print("Test score:", model.score(X_test, y_test))Key takeaways
- Fit happens on training data.
- The same transformations are applied to test data.
- Pipelines reduce bugs and make code cleaner.
Visualize it
flowchart LR A["Raw DataFrame"] --> B["Numeric columns"] A --> C["Categorical columns"] B --> D["Impute (median) -> Scale"] C --> E["Impute (most frequent) -> OneHotEncode"] D --> F["ColumnTransformer"] E --> F F --> G["Model (fit on train)"]
🧪 Try It Yourself
Exercise 1 – Chain steps with Pipeline
Tab indents. Press Escape, then Tab, to leave the editor. Control plus Enter runs the code; Control plus Shift plus Enter submits it.
Press Run to see what your code prints.Exercise 2 – Route columns with ColumnTransformer
Tab indents. Press Escape, then Tab, to leave the editor. Control plus Enter runs the code; Control plus Shift plus Enter submits it.
Press Run to see what your code prints.Exercise 3 – Fit a full model pipeline
Tab indents. Press Escape, then Tab, to leave the editor. Control plus Enter runs the code; Control plus Shift plus Enter submits it.
Press Run to see what your code prints.Check your understanding
What does ColumnTransformer let you do in the lesson example?
The num transformer imputes and scales age and score, while the cat transformer imputes and one-hot encodes city.
Show answer
A — Apply different preprocessing to numeric columns and categorical columns — The num transformer imputes and scales age and score, while the cat transformer imputes and one-hot encodes city.
Which strategies does the lesson use in its two imputers?
SimpleImputer(strategy="median") fills numeric gaps, and strategy="most_frequent" fills the missing city.
Show answer
A — median for numeric, most_frequent for categorical — SimpleImputer(strategy="median") fills numeric gaps, and strategy="most_frequent" fills the missing city.
On which data does model.fit(X_train, y_train) learn the imputer medians, scaler statistics and encoder categories?
Fit happens on training data, and the same fitted transformations are applied to the test data, which avoids leakage.
Show answer
A — Training data only — Fit happens on training data, and the same fitted transformations are applied to the test data, which avoids leakage.
Which three benefits does the lesson list for pipelines?
Pipelines bundle preprocessing and the model, reducing bugs and making the code cleaner.
Show answer
A — Avoid leakage, keep train and test preprocessing consistent, make training reproducible — Pipelines bundle preprocessing and the model, reducing bugs and making the code cleaner.
Next
Pipelines wire together the steps you've learned in this phase — round it out with Feature Engineering Basics to build richer inputs before they enter the pipeline.