Skip to content
Python Central Hub

Multi-Layer Perceptron (MLP)

A single perceptron can only draw a straight line through your data — it can’t even solve the classic XOR problem. It turns out the fix is almost embarrassingly simple: stack more than one layer of neurons. That’s a Multi-Layer Perceptron, and it’s the ancestor of every deep network you’ll meet later in this module.

What an MLP is

An MLP is composed of one (passthrough) input layer, one or more hidden layers of TLUs, and a final output layer. Every layer except the output layer is fully connected to the next — every neuron in the previous layer feeds every neuron in the next one. This is a fully connected (dense) layer. Because signal only flows forward, from inputs to outputs, this is called a feedforward neural network (FNN). Stack enough hidden layers and you get a deep neural network (DNN) — the “deep” in Deep Learning.

An MLP is a feed-forward neural network with:

  • an input layer
  • one or more hidden layers
  • an output layer
diagram Diagram mermaid

Why hidden layers matter

Hidden layers + non-linear activations allow the model to learn:

  • curves
  • interactions
  • complex decision boundaries

Typical uses

MLP works best on:

  • tabular data (sometimes)
  • small image/text tasks (but CNNs/Transformers are usually better)

How training works (a preview)

For decades, nobody could figure out how to train an MLP. The breakthrough — still used today — is backpropagation (Phase 2 covers it in depth):

  1. Forward pass: feed a batch of data through the network, layer by layer, and keep every intermediate result.
  2. Measure the loss: compare the network’s output to the target with a loss function.
  3. Backward pass: apply the chain rule, working backward from the output layer to the input layer, to work out how much each weight contributed to the error.
  4. Gradient Descent step: nudge every weight a little in the direction that reduces the error, and repeat.

Two details make this actually work:

  • Weights must be initialized randomly, not to zero — otherwise every neuron in a layer stays identical forever, and the layer behaves like it has just one neuron.
  • Hidden layers need a non-linear activation (Phase 1’s next page). The step function used by the original perceptron has zero gradient almost everywhere, so the authors of backpropagation swapped it for the smooth logistic (sigmoid) function instead.

Building MLPs with tf.keras’s Sequential API

Here’s a classification MLP with two hidden layers, trained on Fashion MNIST (Keras’s drop-in replacement for MNIST) — straight out of the book:

A classification MLP with the Sequential API
from tensorflow import keras
 
fashion_mnist = keras.datasets.fashion_mnist
(X_train_full, y_train_full), (X_test, y_test) = fashion_mnist.load_data()
 
# Pixels arrive as 0-255 integers; scale to 0-1 floats and hold out a
# small validation set from the training data.
X_valid, X_train = X_train_full[:5000] / 255.0, X_train_full[5000:] / 255.0
y_valid, y_train = y_train_full[:5000], y_train_full[5000:]
 
model = keras.Sequential([
    keras.layers.Flatten(input_shape=[28, 28]),   # 28x28 image -> 784-long vector
    keras.layers.Dense(300, activation="relu"),   # hidden layer 1
    keras.layers.Dense(100, activation="relu"),   # hidden layer 2
    keras.layers.Dense(10, activation="softmax"), # 10 output classes
])
 
model.compile(loss="sparse_categorical_crossentropy",
              optimizer="sgd",
              metrics=["accuracy"])
 
history = model.fit(X_train, y_train, epochs=10,
                     validation_data=(X_valid, y_valid))
 
model.evaluate(X_test, y_test)
A classification MLP with the Sequential API
from tensorflow import keras
 
fashion_mnist = keras.datasets.fashion_mnist
(X_train_full, y_train_full), (X_test, y_test) = fashion_mnist.load_data()
 
# Pixels arrive as 0-255 integers; scale to 0-1 floats and hold out a
# small validation set from the training data.
X_valid, X_train = X_train_full[:5000] / 255.0, X_train_full[5000:] / 255.0
y_valid, y_train = y_train_full[:5000], y_train_full[5000:]
 
model = keras.Sequential([
    keras.layers.Flatten(input_shape=[28, 28]),   # 28x28 image -> 784-long vector
    keras.layers.Dense(300, activation="relu"),   # hidden layer 1
    keras.layers.Dense(100, activation="relu"),   # hidden layer 2
    keras.layers.Dense(10, activation="softmax"), # 10 output classes
])
 
model.compile(loss="sparse_categorical_crossentropy",
              optimizer="sgd",
              metrics=["accuracy"])
 
history = model.fit(X_train, y_train, epochs=10,
                     validation_data=(X_valid, y_valid))
 
model.evaluate(X_test, y_test)

Regression MLPs look almost identical — the output layer just has a single neuron and no activation, so it’s free to predict any real value:

A regression MLP
from tensorflow import keras
from sklearn.datasets import fetch_california_housing
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
 
housing = fetch_california_housing()
X_train_full, X_test, y_train_full, y_test = train_test_split(housing.data, housing.target)
X_train, X_valid, y_train, y_valid = train_test_split(X_train_full, y_train_full)
 
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_valid = scaler.transform(X_valid)
X_test = scaler.transform(X_test)
 
model = keras.Sequential([
    keras.layers.Dense(30, activation="relu", input_shape=X_train.shape[1:]),
    keras.layers.Dense(1),   # no activation: any real-valued prediction is allowed
])
model.compile(loss="mean_squared_error", optimizer="sgd")
 
history = model.fit(X_train, y_train, epochs=20, validation_data=(X_valid, y_valid))
mse_test = model.evaluate(X_test, y_test)
A regression MLP
from tensorflow import keras
from sklearn.datasets import fetch_california_housing
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
 
housing = fetch_california_housing()
X_train_full, X_test, y_train_full, y_test = train_test_split(housing.data, housing.target)
X_train, X_valid, y_train, y_valid = train_test_split(X_train_full, y_train_full)
 
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_valid = scaler.transform(X_valid)
X_test = scaler.transform(X_test)
 
model = keras.Sequential([
    keras.layers.Dense(30, activation="relu", input_shape=X_train.shape[1:]),
    keras.layers.Dense(1),   # no activation: any real-valued prediction is allowed
])
model.compile(loss="mean_squared_error", optimizer="sgd")
 
history = model.fit(X_train, y_train, epochs=20, validation_data=(X_valid, y_valid))
mse_test = model.evaluate(X_test, y_test)

Mini-checkpoint

If you remove non-linear activations from all hidden layers, what happens?

  • The whole network behaves like a linear model.

Visualize it

An MLP stacks layers of neurons, each fully connected to the next.

diagram MLP network topology mermaid
An input layer of three nodes connects densely to a hidden layer of four nodes, which connects densely to a single output node.

Here’s the same idea with real numbers flowing through it: three inputs feed a hidden layer of three ReLU neurons, which feed a single output neuron. Edge thickness shows the size of the weight; green means positive, red means negative:

sketch A tiny feed-forward network with weighted edges p5.js
Watch one forward pass ripple through the network: input signals travel to the hidden layer, ReLU fires, then the result travels on to the output.

Next

You now know how to stack layers into an MLP and build one with tf.keras. Next up: Activation Functions (ReLU, Sigmoid, Softmax) — the non-linear pieces that make all of this actually work.

🧪 Try It Yourself

Exercise 1 – Stack Hidden Layers

Exercise 2 – A Regression Output Layer

Exercise 3 – Compile for Multiclass Classification

If this helped you, consider buying me a coffee ☕

Buy me a coffee

Was this page helpful?

Let us know how we did