Simple Linear Regression

The model

Simple linear regression predicts a number from one feature:

ŷ = w·x + bŷ = w·x + b

ww is the slope (how much y changes per unit x)
bb is the intercept (y when x = 0)

diagram Diagram mermaid

  flowchart LR
  x[Single feature x] -->|w, b| yhat[Prediction ŷ]

Intuition

If w = 200w = 200 and xx is house size in sqft:

increasing size by 1 sqft increases predicted price by 200 (units of currency)

Scikit-learn example

Simple linear regression

import numpy as np
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error
 
# X must be 2D in scikit-learn
X = np.array([500, 700, 800, 1000, 1200]).reshape(-1, 1)
y = np.array([100, 150, 170, 210, 240])
 
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
 
model = LinearRegression()
model.fit(X_train, y_train)
 
pred = model.predict(X_test)
print("w (slope):", model.coef_[0])
print("b (intercept):", model.intercept_)
print("MSE:", mean_squared_error(y_test, pred))

Simple linear regression

import numpy as np
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error
 
# X must be 2D in scikit-learn
X = np.array([500, 700, 800, 1000, 1200]).reshape(-1, 1)
y = np.array([100, 150, 170, 210, 240])
 
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
 
model = LinearRegression()
model.fit(X_train, y_train)
 
pred = model.predict(X_test)
print("w (slope):", model.coef_[0])
print("b (intercept):", model.intercept_)
print("MSE:", mean_squared_error(y_test, pred))

Pitfalls

outliers can strongly affect the line
if the relationship is non-linear, the line underfits

Mini-checkpoint

Plot x vs y. Does it look roughly linear?

If yes, start here.
If no, consider polynomial regression or other models.

Visualize it

Fitting a simple linear regression means finding the slope and intercept of the line that sits as close as possible to all the points. Here the line starts flat and adjusts itself step by step to minimize the total distance to the data — click to generate a new dataset:

sketch Fitting the line of best fit p5.js

The line adjusts its slope and intercept to minimize the distance to the data points.

🧪 Try It Yourself

Exercise 1 – Train-Test Split

Exercise 2 – Fit a Linear Model

Exercise 3 – Evaluate with MSE

Continue to Multiple Linear Regression — extend the same idea to more than one feature, and see how the Normal Equation scales.

If this helped you, consider buying me a coffee ☕

Buy me a coffee