What you’ll learn

• why a model that runs fine on a server can be unusable on a phone
• how the TFLite converter turns a SavedModel into a compact .tflite.tflite FlatBuffer, pruning and fusing operations along the way
• how quantization shrinks weights from 32-bit floats down to 8-bit integers — and what accuracy you trade away for it
• how to run inference on-device with the TFLite interpreter
• why some accelerator chips (like the Edge TPU) require full integer quantization, and how TensorFlow.js targets a third runtime — the browser — instead of a phone

Why mobile needs a different model format

A model that’s perfectly fine on a server — gigabytes of weights, 32-bit floats everywhere — can be a disaster on a phone: too slow to download, too much RAM, too much battery drained per prediction, and a phone that gets uncomfortably warm. TensorFlow Lite (TFLite) exists to shrink models along three axes at once: smaller download and memory footprint, fewer computations per prediction, and support for device-specific constraints (some accelerator chips only support integer math).

  flowchart LR
  A["Train (tf.keras)"] --> B["Export SavedModel"]
  B --> C["TFLite converter → .tflite FlatBuffer"]
  C --> D["Quantize weights (float32 → int8)"]
  D --> E["TFLite interpreter on-device"]

Converting a model

The converter takes a SavedModel (or a tf.kerastf.keras model directly) and produces a .tflite.tflite file — a FlatBuffer, a serialization format designed to be loaded straight into RAM with no parsing step, which keeps load time and memory overhead low:

import tensorflow as tf
 
converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_path)
# or: converter = tf.lite.TFLiteConverter.from_keras_model(model)
tflite_model = converter.convert()
 
with open("converted_model.tflite", "wb") as f:
    f.write(tflite_model)

import tensorflow as tf
 
converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_path)
# or: converter = tf.lite.TFLiteConverter.from_keras_model(model)
tflite_model = converter.convert()
 
with open("converted_model.tflite", "wb") as f:
    f.write(tflite_model)

While it converts, the tool also optimizes: it prunes operations that only exist for training (like the gradient computation), simplifies expressions (3*a + 4*a + 5*a3*a + 4*a + 5*a becomes (3+4+5)*a(3+4+5)*a), and fuses operations where it can — for example, folding a Batch Normalization layer directly into the previous layer’s multiply-and-add.

Quantization: fewer bits per weight

Beyond structural optimization, TFLite can also shrink the numbers themselves. Dropping from 32-bit floats to 16-bit halves the model size for a small accuracy cost. Going further — down to 8-bit integers — gives a 4x size reduction over 32-bit floats, using post-training quantization: find the largest absolute weight value mm, then map the float range [-m, +m][-m, +m] onto the integer range [-127, +127][-127, +127]:

import numpy as np
 
weights = np.array([-1.5, 0.0, 0.8])
m = np.max(np.abs(weights))          # 1.5
 
quantized = np.round(weights / m * 127).astype(int)
print(quantized)
# [-127    0   68]

import numpy as np
 
weights = np.array([-1.5, 0.0, 0.8])
m = np.max(np.abs(weights))          # 1.5
 
quantized = np.round(weights / m * 127).astype(int)
print(quantized)
# [-127    0   68]

-1.5-1.5 becomes -127-127, 0.00.0 stays 00, and 0.80.8 becomes 6868 — because 0.80.8 is about 53% of the way from 00 to 1.51.5, and 6868 is about 53% of the way from 00 to 127127. In TFLite, this one-liner turns the feature on:

converter.optimizations = [tf.lite.Optimize.OPTIMIZE_FOR_SIZE]
tflite_quant_model = converter.convert()

converter.optimizations = [tf.lite.Optimize.OPTIMIZE_FOR_SIZE]
tflite_quant_model = converter.convert()

This shrinks download and storage size dramatically. But by default, the quantized weights get converted back to floats before they’re used in a computation — so it doesn’t reduce RAM usage or speed up computation, only size on disk. To actually speed up inference and cut power use, you also need to quantize the activations, so every computation happens in integer math end to end. This requires a calibration step: TFLite runs a small representative sample of your training data through the model to measure the range of activation values it needs to quantize.

Running inference on-device

Once the model is loaded onto the device, the lightweight TFLite interpreter (not a full TensorFlow install) executes it:

import numpy as np
import tensorflow as tf
 
interpreter = tf.lite.Interpreter(model_path="converted_model.tflite")
interpreter.allocate_tensors()
 
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
 
interpreter.set_tensor(input_details[0]["index"], X_new.astype(np.float32))
interpreter.invoke()
predictions = interpreter.get_tensor(output_details[0]["index"])

import numpy as np
import tensorflow as tf
 
interpreter = tf.lite.Interpreter(model_path="converted_model.tflite")
interpreter.allocate_tensors()
 
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
 
interpreter.set_tensor(input_details[0]["index"], X_new.astype(np.float32))
interpreter.invoke()
predictions = interpreter.get_tensor(output_details[0]["index"])

allocate_tensors()allocate_tensors() reserves memory for every tensor in the graph up front — no dynamic allocation during inference, which keeps latency predictable, an important property on constrained hardware.

Beyond phones: running in the browser with TensorFlow.js

TFLite targets phones and embedded devices, but there’s a third deployment target that needs its own lightweight format: a model running directly inside a user’s browser tab, with no server round-trip and no app install. That’s what TensorFlow.js is for — a JavaScript library that loads a converted model and runs inference on the client’s own CPU or GPU (via WebGL).

This is worth reaching for when:

• your users have unreliable connectivity (a hiking or field-work app can’t depend on a round-trip to your server for every prediction)
• you need the lowest possible latency — an in-browser game or live camera filter can’t afford a network hop per frame
• the input is sensitive user data that should never leave their machine — running inference client-side is a simple form of privacy-by-design

The tensorflowjs_convertertensorflowjs_converter tool (a separate pip install tensorflowjspip install tensorflowjs package) turns a SavedModel or Keras model into the TF.js Layers format: a model.jsonmodel.json describing the architecture plus a handful of sharded binary weight files, sized so a browser can download and cache them efficiently:

# $ pip install tensorflowjs
# $ tensorflowjs_converter --input_format=tf_saved_model \
#     saved_model_path/ web_model/

# $ pip install tensorflowjs
# $ tensorflowjs_converter --input_format=tf_saved_model \
#     saved_model_path/ web_model/

On the page itself, loading and predicting is a handful of lines of JavaScript, not Python — a genuinely different runtime from everything else in this phase:

import * as tf from "@tensorflow/tfjs";
 
const model = await tf.loadLayersModel("https://example.com/web_model/model.json");
const image = tf.browser.fromPixels(webcamElement);
const prediction = model.predict(image.expandDims(0));

import * as tf from "@tensorflow/tfjs";
 
const model = await tf.loadLayersModel("https://example.com/web_model/model.json");
const image = tf.browser.fromPixels(webcamElement);
const prediction = model.predict(image.expandDims(0));

TFLite and TF.js solve the same underlying problem — get a trained model close to where its predictions are consumed — for two different runtimes: a phone’s native code versus a browser’s JavaScript engine. Pick based on where your users actually are, not which one sounds more advanced.

Visualize it

Watch how a handful of continuous weight values collapse onto the same small set of 8-bit integer buckets — the gaps between buckets are exactly where quantization loses information:

Mini-checkpoint

• .tflite.tflite = a FlatBuffer: prunes training-only ops, fuses what it can, loads straight into RAM.
• Post-training weight quantization → smaller download, no runtime speedup.
• Post-training weight + activation quantization (needs calibration data) → smaller and faster, at the cost of accuracy.
• If accuracy drops too much, train with fake quantization ops baked in (quantization-aware training) instead of quantizing after the fact.
• Different runtime, different tool: TFLite → phones/embedded (and full quantization is required for chips like the Edge TPU); TensorFlow.js → the browser, no install and no server round-trip.

🧪 Try It Yourself

Exercise 1 – Convert a Keras model to TFLite

Exercise 2 – Quantize weights to int8 by hand

Exercise 3 – Dequantize back to floats

Next

Continue to Limitations and the Future of Deep Learning — you’ve now taken model training all the way from a custom GradientTapeGradientTape loop, through tuning and distributing it across GPUs, to serving predictions over a REST API and running the same model on a phone in someone’s pocket. Before you close this module, it’s worth a last look at what all of that still can’t do.