Skip to content
Python Central Hub

Polynomial Regression

Why polynomial regression exists

Sometimes the relationship is curved:

  • learning curves
  • growth/decay
  • diminishing returns

A straight line underfits.

Polynomial regression keeps a linear model but expands features:

  • x → [x, x², x³, …]

Picture intuition

false

  flowchart LR
  x[x] --> p[PolynomialFeatures]
  p --> Xexp[Expanded features: x, x², x³]
  Xexp --> lin[Linear Regression]
  lin --> yhat[ŷ]

false

Scikit-learn example

Polynomial regression
import numpy as np
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model import LinearRegression
 
X = np.array([0, 1, 2, 3, 4]).reshape(-1, 1)
y = np.array([1, 2, 5, 10, 17])  # roughly quadratic
 
poly_model = Pipeline(
    steps=[
        ("poly", PolynomialFeatures(degree=2, include_bias=False)),
        ("lin", LinearRegression()),
    ]
)
 
poly_model.fit(X, y)
print(poly_model.predict([[5]]))
Polynomial regression
import numpy as np
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model import LinearRegression
 
X = np.array([0, 1, 2, 3, 4]).reshape(-1, 1)
y = np.array([1, 2, 5, 10, 17])  # roughly quadratic
 
poly_model = Pipeline(
    steps=[
        ("poly", PolynomialFeatures(degree=2, include_bias=False)),
        ("lin", LinearRegression()),
    ]
)
 
poly_model.fit(X, y)
print(poly_model.predict([[5]]))

Overfitting warning

Higher degrees can fit noise.

Use:

  • validation
  • regularization (Ridge/Lasso)

Mini-checkpoint

Try degrees 1, 2, 3.

  • does validation error improve?
  • does it start getting worse at high degree?

If this helped you, consider buying me a coffee ☕

Buy me a coffee

Was this page helpful?

Let us know how we did