Skip to content
Python Central Hub

First Example: Predicting House Prices (Regression)

The last two examples predicted a category. This one predicts a number — the median price of a house — which changes the output layer, the loss function, and ( because there’s so little data) how you validate the model at all.

The dataset: few samples, mismatched scales

This dataset has only 506 data points total — 404 for training, 102 for testing — each with 13 numerical features about a Boston-area suburb in the 1970s (crime rate, rooms per dwelling, distance to employment centers, and so on). Two things make this example different from IMDB and Reuters:

  • Very little data — 404 samples is tiny by deep learning standards.
  • Mismatched feature scales — one column might range 0–1, another 1–12, another 0–100.
Loading the housing dataset
from tensorflow.keras.datasets import boston_housing
 
(train_data, train_targets), (test_data, test_targets) = boston_housing.load_data()
 
print(train_data.shape)   # (404, 13)
print(test_data.shape)    # (102, 13)
print(train_targets[:5])  # e.g. [15.2, 42.3, 50.0, 21.1, 17.7] -- prices in $1000s
Loading the housing dataset
from tensorflow.keras.datasets import boston_housing
 
(train_data, train_targets), (test_data, test_targets) = boston_housing.load_data()
 
print(train_data.shape)   # (404, 13)
print(test_data.shape)    # (102, 13)
print(train_targets[:5])  # e.g. [15.2, 42.3, 50.0, 21.1, 17.7] -- prices in $1000s

Preparing the data: feature-wise normalization

Feeding a network features on wildly different scales makes learning needlessly hard. The standard fix: for each column, subtract its mean and divide by its standard deviation, so every feature ends up centered at 0 with a standard deviation of 1:

Normalizing every feature
mean = train_data.mean(axis=0)
train_data -= mean
std = train_data.std(axis=0)
train_data /= std
 
test_data -= mean   # reuse the TRAINING mean/std -- never compute stats from test data
test_data /= std
Normalizing every feature
mean = train_data.mean(axis=0)
train_data -= mean
std = train_data.std(axis=0)
train_data /= std
 
test_data -= mean   # reuse the TRAINING mean/std -- never compute stats from test data
test_data /= std

The test set is normalized using the training set’s mean and standard deviation — using anything computed from the test data, even something this simple, leaks information you’re not supposed to have yet.

Building the model: a linear output layer

With so few samples, a small model helps avoid overfitting — two 64-unit hidden layers is plenty here:

A regression model, wrapped in a function
from tensorflow import keras
from tensorflow.keras import layers
 
def build_model():
    model = keras.Sequential([
        layers.Dense(64, activation="relu"),
        layers.Dense(64, activation="relu"),
        layers.Dense(1),   # no activation -- a purely linear output
    ])
    model.compile(optimizer="rmsprop", loss="mse", metrics=["mae"])
    return model
A regression model, wrapped in a function
from tensorflow import keras
from tensorflow.keras import layers
 
def build_model():
    model = keras.Sequential([
        layers.Dense(64, activation="relu"),
        layers.Dense(64, activation="relu"),
        layers.Dense(1),   # no activation -- a purely linear output
    ])
    model.compile(optimizer="rmsprop", loss="mse", metrics=["mae"])
    return model
  • The output layer has 1 unit and no activation function. Any activation (like sigmoidsigmoid) would squash the output into a fixed range — but a house price can be any real number, so the last layer stays purely linear (dot(input, W) + bdot(input, W) + b).
  • msemse (mean squared error) is the standard regression loss.
  • maemae (mean absolute error) is tracked as a metric because it’s easy to interpret directly: an MAE of 2.5 here means predictions are off by about $2,500 on average.
  • build_model()build_model() is wrapped in a function because K-fold validation, next, needs a fresh, identically-initialized model for every fold.
diagram Classification vs. regression output layers mermaid
Classification ends in sigmoid or softmax to produce probabilities; regression ends in a single linear unit that can output any real value.

Why K-fold validation

With only 404 training samples, a single train/validation split would leave a validation set of maybe 100 examples — small enough that the validation score would swing wildly depending on which 100 examples happened to land there. K-fold cross-validation fixes this: split the data into kk chunks, train kk separate models (each holding out a different chunk for validation), and average the results.

K-fold validation
import numpy as np
 
k = 4
num_val_samples = len(train_data) // k
num_epochs = 100
all_scores = []
 
for i in range(k):
    val_data = train_data[i * num_val_samples: (i + 1) * num_val_samples]
    val_targets = train_targets[i * num_val_samples: (i + 1) * num_val_samples]
 
    partial_train_data = np.concatenate(
        [train_data[:i * num_val_samples], train_data[(i + 1) * num_val_samples:]], axis=0)
    partial_train_targets = np.concatenate(
        [train_targets[:i * num_val_samples], train_targets[(i + 1) * num_val_samples:]], axis=0)
 
    model = build_model()
    model.fit(partial_train_data, partial_train_targets,
              epochs=num_epochs, batch_size=16, verbose=0)
    val_mse, val_mae = model.evaluate(val_data, val_targets, verbose=0)
    all_scores.append(val_mae)
 
print(all_scores)          # e.g. [2.1, 3.1, 2.6, 2.4] -- scores vary per fold
print(np.mean(all_scores)) # ~2.55 -- a far more reliable estimate than any single fold
K-fold validation
import numpy as np
 
k = 4
num_val_samples = len(train_data) // k
num_epochs = 100
all_scores = []
 
for i in range(k):
    val_data = train_data[i * num_val_samples: (i + 1) * num_val_samples]
    val_targets = train_targets[i * num_val_samples: (i + 1) * num_val_samples]
 
    partial_train_data = np.concatenate(
        [train_data[:i * num_val_samples], train_data[(i + 1) * num_val_samples:]], axis=0)
    partial_train_targets = np.concatenate(
        [train_targets[:i * num_val_samples], train_targets[(i + 1) * num_val_samples:]], axis=0)
 
    model = build_model()
    model.fit(partial_train_data, partial_train_targets,
              epochs=num_epochs, batch_size=16, verbose=0)
    val_mse, val_mae = model.evaluate(val_data, val_targets, verbose=0)
    all_scores.append(val_mae)
 
print(all_scores)          # e.g. [2.1, 3.1, 2.6, 2.4] -- scores vary per fold
print(np.mean(all_scores)) # ~2.55 -- a far more reliable estimate than any single fold

Individual folds can disagree by a dollar or more per house — the average across folds is what you should trust.

Choosing the number of epochs

Training longer (say 500 epochs) while recording the validation MAE at every epoch, for every fold, lets you plot the average validation MAE curve and read off where it stops improving:

Recording per-epoch validation history across folds
all_mae_histories = []
for i in range(k):
    # ... same fold split as above ...
    model = build_model()
    history = model.fit(partial_train_data, partial_train_targets,
                         validation_data=(val_data, val_targets),
                         epochs=500, batch_size=16, verbose=0)
    all_mae_histories.append(history.history["val_mae"])
 
average_mae_history = [
    np.mean([fold[epoch] for fold in all_mae_histories]) for epoch in range(500)
]
Recording per-epoch validation history across folds
all_mae_histories = []
for i in range(k):
    # ... same fold split as above ...
    model = build_model()
    history = model.fit(partial_train_data, partial_train_targets,
                         validation_data=(val_data, val_targets),
                         epochs=500, batch_size=16, verbose=0)
    all_mae_histories.append(history.history["val_mae"])
 
average_mae_history = [
    np.mean([fold[epoch] for fold in all_mae_histories]) for epoch in range(500)
]

In the book, this curve flattens out around epoch 120–140 — training past that point just overfits.

Training the final model

Once you’ve picked a good number of epochs, train one last model on all the training data and check it against the test set:

Training the final production model
model = build_model()
model.fit(train_data, train_targets, epochs=130, batch_size=16, verbose=0)
test_mse_score, test_mae_score = model.evaluate(test_data, test_targets)
print(test_mae_score)   # roughly 2.5 -- off by about $2,500 on average
Training the final production model
model = build_model()
model.fit(train_data, train_targets, epochs=130, batch_size=16, verbose=0)
test_mse_score, test_mae_score = model.evaluate(test_data, test_targets)
print(test_mae_score)   # roughly 2.5 -- off by about $2,500 on average
Predicting a price
predictions = model.predict(test_data)
print(predictions[0])   # e.g. array([9.99], dtype=float32) -- ~$10,000
Predicting a price
predictions = model.predict(test_data)
print(predictions[0])   # e.g. array([9.99], dtype=float32) -- ~$10,000

Mini-checkpoint

Why does the model’s final DenseDense layer have no activation function?

  • Applying an activation like sigmoidsigmoid or softmaxsoftmax would constrain the output to a fixed range (like 0–1). A house price can be any real number, so the last layer stays a plain linear transformation.

Next

You’ve now built all three of the classic vector-data workflows — binary classification, multiclass classification, and regression. Continue to Phase 2: Training Deep Neural Networks to see how backpropagation, optimizers, and regularization scale these same ideas up to bigger, deeper networks.

🧪 Try It Yourself

Exercise 1 – Normalize Features by Hand

Exercise 2 – A Linear Output Layer for Regression

Exercise 3 – Average K-Fold Scores

If this helped you, consider buying me a coffee ☕

Buy me a coffee

Was this page helpful?

Let us know how we did