Skip to content
Python Central Hub

Building Neural Networks with Keras (Sequential and Functional API)

You’ve already built a couple of keras.Sequentialkeras.Sequential models in this phase. That covers a lot of ground, but not everything — what if two layers need to skip past a third, or a model needs two inputs, or its forward pass involves a loop? Keras gives you two more ways to describe a model, plus the tools to actually run, save, and babysit training once the architecture is built.

Three ways to build a model

  • Sequential API — a plain stack of layers, one feeding the next. Simple, but every layer must have exactly one input and one output, in a straight line.
  • Functional API — you call each layer like a function on a previous layer’s output, so you can branch, merge, and reuse layers freely. Still declarative: you describe the graph first, then feed it data.
  • Subclassing API — you write the forward pass yourself, in ordinary Python, inside a call()call() method. Imperative, and the most flexible of the three.
diagram Three ways to build a Keras model mermaid
Sequential stacks layers in a line; Functional connects layers like function calls, allowing branches; Subclassing writes the forward pass as code.

The Functional API: Wide & Deep

A Wide & Deep network (Cheng et al., 2016) sends the raw input straight to the output through a short “wide” path, and through a stack of hidden layers via a “deep” path, then concatenates both before the final prediction. The wide path can memorize simple rules; the deep path can learn more complex patterns — and a plain SequentialSequential stack can’t express this, because the input has two separate places to go.

A Wide & Deep model with the Functional API
from tensorflow import keras
 
input_ = keras.layers.Input(shape=[8])
hidden1 = keras.layers.Dense(30, activation="relu")(input_)
hidden2 = keras.layers.Dense(30, activation="relu")(hidden1)
concat = keras.layers.Concatenate()([input_, hidden2])
output = keras.layers.Dense(1)(concat)
 
model = keras.Model(inputs=[input_], outputs=[output])
model.compile(loss="mse", optimizer="sgd")
A Wide & Deep model with the Functional API
from tensorflow import keras
 
input_ = keras.layers.Input(shape=[8])
hidden1 = keras.layers.Dense(30, activation="relu")(input_)
hidden2 = keras.layers.Dense(30, activation="relu")(hidden1)
concat = keras.layers.Concatenate()([input_, hidden2])
output = keras.layers.Dense(1)(concat)
 
model = keras.Model(inputs=[input_], outputs=[output])
model.compile(loss="mse", optimizer="sgd")

Notice hidden1 = keras.layers.Dense(30, activation="relu")(input_)hidden1 = keras.layers.Dense(30, activation="relu")(input_) — you create the layer, then immediately call it like a function, passing in the previous layer’s output. Keras is just recording how the layers connect; no data is processed yet. That’s the “Functional” in Functional API.

You can also give a model multiple inputs — say five features through the wide path and six (possibly overlapping) through the deep path:

A Wide & Deep model with two separate inputs
from tensorflow import keras
 
input_A = keras.layers.Input(shape=[5], name="wide_input")
input_B = keras.layers.Input(shape=[6], name="deep_input")
hidden1 = keras.layers.Dense(30, activation="relu")(input_B)
hidden2 = keras.layers.Dense(30, activation="relu")(hidden1)
concat = keras.layers.concatenate([input_A, hidden2])
output = keras.layers.Dense(1, name="output")(concat)
 
model = keras.Model(inputs=[input_A, input_B], outputs=[output])
model.compile(loss="mse", optimizer=keras.optimizers.SGD(learning_rate=1e-3))
 
# fit/evaluate/predict now take a tuple or list, one array per input
# history = model.fit((X_train_A, X_train_B), y_train, epochs=20)
A Wide & Deep model with two separate inputs
from tensorflow import keras
 
input_A = keras.layers.Input(shape=[5], name="wide_input")
input_B = keras.layers.Input(shape=[6], name="deep_input")
hidden1 = keras.layers.Dense(30, activation="relu")(input_B)
hidden2 = keras.layers.Dense(30, activation="relu")(hidden1)
concat = keras.layers.concatenate([input_A, hidden2])
output = keras.layers.Dense(1, name="output")(concat)
 
model = keras.Model(inputs=[input_A, input_B], outputs=[output])
model.compile(loss="mse", optimizer=keras.optimizers.SGD(learning_rate=1e-3))
 
# fit/evaluate/predict now take a tuple or list, one array per input
# history = model.fit((X_train_A, X_train_B), y_train, epochs=20)

Multiple outputs work the same way — useful when a task naturally has more than one target, or when an auxiliary output (attached to an earlier layer) acts as a regularizer that forces the underlying layers to learn something useful on their own:

Adding an auxiliary output
from tensorflow import keras
 
output = keras.layers.Dense(1, name="main_output")(concat)
aux_output = keras.layers.Dense(1, name="aux_output")(hidden2)
model = keras.Model(inputs=[input_A, input_B], outputs=[output, aux_output])
 
# each output needs its own loss; give the main one more weight
model.compile(loss=["mse", "mse"], loss_weights=[0.9, 0.1], optimizer="sgd")
Adding an auxiliary output
from tensorflow import keras
 
output = keras.layers.Dense(1, name="main_output")(concat)
aux_output = keras.layers.Dense(1, name="aux_output")(hidden2)
model = keras.Model(inputs=[input_A, input_B], outputs=[output, aux_output])
 
# each output needs its own loss; give the main one more weight
model.compile(loss=["mse", "mse"], loss_weights=[0.9, 0.1], optimizer="sgd")

The Subclassing API

Both APIs above are declarative — the whole architecture is a static graph before any data flows through it, which is exactly why Keras can validate shapes early, draw the model, and save it easily. But some architectures involve loops, branching ifif statements, or shapes that change at runtime. For that (or just a more imperative coding style), subclass keras.Modelkeras.Model directly: create the layers in __init____init__, and use them in call()call().

The same Wide & Deep model, as a subclass
from tensorflow import keras
 
class WideAndDeepModel(keras.Model):
    def __init__(self, units=30, activation="relu", **kwargs):
        super().__init__(**kwargs)
        self.hidden1 = keras.layers.Dense(units, activation=activation)
        self.hidden2 = keras.layers.Dense(units, activation=activation)
        self.main_output = keras.layers.Dense(1)
        self.aux_output = keras.layers.Dense(1)
 
    def call(self, inputs):
        input_A, input_B = inputs
        hidden1 = self.hidden1(input_B)
        hidden2 = self.hidden2(hidden1)
        concat = keras.layers.concatenate([input_A, hidden2])
        main_output = self.main_output(concat)
        aux_output = self.aux_output(hidden2)
        return main_output, aux_output
 
model = WideAndDeepModel()
model.compile(loss=["mse", "mse"], loss_weights=[0.9, 0.1], optimizer="sgd")
The same Wide & Deep model, as a subclass
from tensorflow import keras
 
class WideAndDeepModel(keras.Model):
    def __init__(self, units=30, activation="relu", **kwargs):
        super().__init__(**kwargs)
        self.hidden1 = keras.layers.Dense(units, activation=activation)
        self.hidden2 = keras.layers.Dense(units, activation=activation)
        self.main_output = keras.layers.Dense(1)
        self.aux_output = keras.layers.Dense(1)
 
    def call(self, inputs):
        input_A, input_B = inputs
        hidden1 = self.hidden1(input_B)
        hidden2 = self.hidden2(hidden1)
        concat = keras.layers.concatenate([input_A, hidden2])
        main_output = self.main_output(concat)
        aux_output = self.aux_output(hidden2)
        return main_output, aux_output
 
model = WideAndDeepModel()
model.compile(loss=["mse", "mse"], loss_weights=[0.9, 0.1], optimizer="sgd")

The trade-off: inside call()call() you can write ordinary Python — forfor loops, ifif statements, anything — but Keras can no longer inspect the architecture ahead of time. model.summary()model.summary() only lists the layers, with no information on how they connect, and the model can’t be cloned or (fully) saved the way a Sequential or Functional model can. Géron’s advice: unless you genuinely need that flexibility, stick to the Sequential or Functional API.

compile, fit, evaluate, predict — same for all three

Whichever API built the model, the training loop looks identical:

The standard Keras workflow
from tensorflow import keras
 
# model = ... (built with any of the three APIs above)
model.compile(loss="sparse_categorical_crossentropy",
              optimizer="sgd",
              metrics=["accuracy"])
 
history = model.fit(X_train, y_train, epochs=10,
                     validation_data=(X_valid, y_valid))
 
model.evaluate(X_test, y_test)
y_pred = model.predict(X_new)
The standard Keras workflow
from tensorflow import keras
 
# model = ... (built with any of the three APIs above)
model.compile(loss="sparse_categorical_crossentropy",
              optimizer="sgd",
              metrics=["accuracy"])
 
history = model.fit(X_train, y_train, epochs=10,
                     validation_data=(X_valid, y_valid))
 
model.evaluate(X_test, y_test)
y_pred = model.predict(X_new)

history.historyhistory.history is a dictionary of per-epoch metrics (lossloss, val_lossval_loss, accuracyaccuracy, …) — the raw data behind every learning-curve plot you’ll draw in this module.

Saving and loading a model

For Sequential and Functional models, saving is a single call — Keras stores the architecture, every layer’s weights, and the optimizer’s state together:

Save and reload a full model
from tensorflow import keras
 
# model.fit(...)
model.save("my_keras_model.h5")
 
# ...later, possibly in a different script
reloaded = keras.models.load_model("my_keras_model.h5")
Save and reload a full model
from tensorflow import keras
 
# model.fit(...)
model.save("my_keras_model.h5")
 
# ...later, possibly in a different script
reloaded = keras.models.load_model("my_keras_model.h5")

load_model()load_model() only works for Sequential and Functional models. A Subclassing model’s architecture lives inside call()call(), which Keras can’t serialize — use model.save_weights(...)model.save_weights(...) / model.load_weights(...)model.load_weights(...) instead, and you’ll need to rebuild the class yourself before restoring the weights into it.

Callbacks: checkpointing and early stopping

If training runs for hours, you don’t want to lose everything to a crash — and you don’t want to babysit the terminal to know when to stop. The fit()fit() method’s callbackscallbacks argument lets Keras call your code at key points during training.

Checkpoint the best model and stop early
from tensorflow import keras
 
checkpoint_cb = keras.callbacks.ModelCheckpoint(
    "my_keras_model.h5", save_best_only=True)
early_stopping_cb = keras.callbacks.EarlyStopping(
    patience=10, restore_best_weights=True)
 
history = model.fit(X_train, y_train, epochs=100,
                     validation_data=(X_valid, y_valid),
                     callbacks=[checkpoint_cb, early_stopping_cb])
Checkpoint the best model and stop early
from tensorflow import keras
 
checkpoint_cb = keras.callbacks.ModelCheckpoint(
    "my_keras_model.h5", save_best_only=True)
early_stopping_cb = keras.callbacks.EarlyStopping(
    patience=10, restore_best_weights=True)
 
history = model.fit(X_train, y_train, epochs=100,
                     validation_data=(X_valid, y_valid),
                     callbacks=[checkpoint_cb, early_stopping_cb])
  • ModelCheckpoint(save_best_only=True)ModelCheckpoint(save_best_only=True) only overwrites the saved file when validation performance improves — so the file on disk is always your best model.
  • EarlyStopping(patience=10)EarlyStopping(patience=10) interrupts training once validation performance hasn’t improved for 10 epochs in a row, and restore_best_weights=Truerestore_best_weights=True rolls the in-memory model back to its best epoch automatically. Since training now stops itself, it’s safe to set epochsepochs to a very large number.

You can write your own callback too, by subclassing keras.callbacks.Callbackkeras.callbacks.Callback and overriding hooks like on_epoch_endon_epoch_end:

A tiny custom callback
from tensorflow import keras
 
class PrintValTrainRatioCallback(keras.callbacks.Callback):
    def on_epoch_end(self, epoch, logs):
        print(f"\nval/train loss ratio: {logs['val_loss'] / logs['loss']:.2f}")
A tiny custom callback
from tensorflow import keras
 
class PrintValTrainRatioCallback(keras.callbacks.Callback):
    def on_epoch_end(self, epoch, logs):
        print(f"\nval/train loss ratio: {logs['val_loss'] / logs['loss']:.2f}")

Mini-checkpoint

You need a model with two separate inputs that get concatenated before a shared hidden layer. Which API do you reach for — and why not Sequential?

  • The Functional API — Sequential only supports a single straight-line stack of layers with one input and one output; it can’t express two inputs merging.

Next

Continue to How Neural Networks Learn (Gradient-Based Optimization) to see how backpropagation and GradientTapeGradientTape actually compute the gradients that compile()compile() and fit()fit() use to update every weight you just wired up.

🧪 Try It Yourself

Exercise 1 – Build a Functional Wide & Deep Model

Exercise 2 – A Subclassed Model

Exercise 3 – Configure Early Stopping

If this helped you, consider buying me a coffee ☕

Buy me a coffee

Was this page helpful?

Let us know how we did