Every deep network — no matter how many layers it has — is built out of one simple idea: take some numbers in, multiply each by a weight, add them up, and decide what to output. That’s it. This page is about that one idea, called a perceptron, and why on its own it’s both powerful and surprisingly limited.

From biological to artificial neurons

Artificial neural networks are loosely inspired by biological neurons: a cell body receives signals through branching dendrites, and if it gets enough signal, it fires its own pulse down its axon to the next neuron. In 1943, Warren McCulloch and Walter Pitts modeled this with a simplified artificial neuron: it takes binary (on/off) inputs and produces a binary output, activating only when enough of its inputs are active. Even with that simple rule, McCulloch and Pitts showed you could wire up networks of these units to compute AND, OR, and NOT — any logical proposition you like.

The perceptron

In 1957, Frank Rosenblatt built on this idea with the perceptron, based on a slightly richer unit called a threshold logic unit (TLU) (also called a linear threshold unit, LTU). Instead of binary on/off values, the TLU’s inputs and output are numbers, and each input connection has its own weight. The perceptron is the simplest neural unit:

1. take inputs
2. compute weighted sum
3. add bias
4. apply activation

Mathematically:

z = w·x + bz = w·x + b

output = activation(z)output = activation(z)

  flowchart LR
  x1[x1] --> N["Σ (weights) + bias"]
  x2[x2] --> N
  x3[x3] --> N
  N --> A[Activation]
  A --> y[Output]

What it can do

A single perceptron can learn a linear decision boundary.

That means:

• it can separate data that is linearly separable
• it cannot solve XOR alone (needs multiple layers)

How a perceptron learns

Rosenblatt’s training rule was inspired by Hebb’s rule: “cells that fire together, wire together.” In practice, the perceptron is fed one training example at a time; for every output that’s wrong, it nudges the weights that would have produced the correct output:

w[i,j] ← w[i,j] + η · (y_j − ŷ_j) · x_iw[i,j] ← w[i,j] + η · (y_j − ŷ_j) · x_i

• ηη (eta) is the learning rate
• y_jy_j is the target output, ŷ_jŷ_j is the perceptron’s actual output
• if the prediction is already correct, nothing changes

If the training data is linearly separable, this algorithm is guaranteed to converge (the Perceptron Convergence Theorem). If it isn’t — like the classic XOR problem — a single perceptron will never find a solution, no matter how long you train it. That limitation is exactly why the next page stacks perceptrons into an MLP.

Perceptron in code

Scikit-learn ships a ready-made PerceptronPerceptron classifier — a single layer of TLUs trained with the rule above:

import numpy as np
from sklearn.datasets import load_iris
from sklearn.linear_model import Perceptron
 
iris = load_iris()
X = iris.data[:, (2, 3)]              # petal length, petal width
y = (iris.target == 0).astype(int)    # Is it Iris setosa?
 
per_clf = Perceptron()
per_clf.fit(X, y)
 
y_pred = per_clf.predict([[2, 0.5]])
print("Predicted class:", y_pred)

import numpy as np
from sklearn.datasets import load_iris
from sklearn.linear_model import Perceptron
 
iris = load_iris()
X = iris.data[:, (2, 3)]              # petal length, petal width
y = (iris.target == 0).astype(int)    # Is it Iris setosa?
 
per_clf = Perceptron()
per_clf.fit(X, y)
 
y_pred = per_clf.predict([[2, 0.5]])
print("Predicted class:", y_pred)

In tf.keras, a single neuron is just a DenseDense layer with one unit. Swapping the hard step function for a smooth sigmoidsigmoid turns this “hard threshold” perceptron into a Logistic Regression-style unit that Gradient Descent can actually train:

from tensorflow import keras
 
# One neuron, two inputs -- the tf.keras equivalent of a TLU.
tlu = keras.layers.Dense(1, activation="sigmoid", input_shape=(2,))
 
model = keras.Sequential([tlu])
model.compile(loss="binary_crossentropy", optimizer="sgd", metrics=["accuracy"])

from tensorflow import keras
 
# One neuron, two inputs -- the tf.keras equivalent of a TLU.
tlu = keras.layers.Dense(1, activation="sigmoid", input_shape=(2,))
 
model = keras.Sequential([tlu])
model.compile(loss="binary_crossentropy", optimizer="sgd", metrics=["accuracy"])

Key terms

• weights: importance of each input
• bias: shift term
• activation: non-linear function

Visualize it

A perceptron is one neuron: it multiplies each input by a weight, adds them up (plus a bias), and passes the total through an activation to decide its output. Thicker edges are bigger weights:

Mini-checkpoint

Why do we need activation functions?

• Without them, the network is just a linear model (even with many layers).

Next

A single perceptron can only draw a straight line. Next up: Multi-Layer Perceptron (MLP) — stacking perceptrons into hidden layers so the network can learn curves and XOR-like patterns.

🧪 Try It Yourself

Exercise 1 – Build a Perceptron

Exercise 2 – Weighted Sum by Hand

Exercise 3 – A Single Keras Neuron