What you’ll learn

• the six kinds of test that apply to ML code, written out, not described
• which bugs each one actually catches — a measured 7 × 7 matrix, not an opinion
• why two of the most commonly written ML tests caught zero of the six bugs
• the one-second test that caught 4 of 6 and localises the failure to the training loop
• why the leaky scaler (accuracy 0.8500, identical to correct) needed a structural test
• how to arrange these tests so CI stays fast enough to actually run

Seven bugs

Each one is a plausible single-line mistake, and each produces a model that runs, returns well-formed probabilities and reports a number.

Note the second row. Leakage through the scaler changed the test accuracy by 0.0000. That is not a mistake in the experiment; on this data, scaling statistics estimated from 4,000 rows instead of 2,800 are almost identical, so the bug is real, undetectable by any metric, and would become dangerous the day the test distribution differs. Hold onto it — it is why the last test in the matrix exists.

Seven tests, measured

figureTest accuracy in brackets. The 'metric floor (0.75)' test caught 5 of 6 bugs.matplotlib

Timings are on this small dataset, where a full fit is 10.3 ms. The ratios are what transfer: a fit on 24 rows costs a fraction of a fit on 2,800, and on a model that takes twenty minutes to train that difference is the whole argument for the memorisation test.

1. The output contract

def test_output_contract(model, X):
    proba = model.predict_proba(X)
    assert proba.shape == (len(X), 2)
    assert np.all((proba >= 0) & (proba <= 1))
    assert np.allclose(proba.sum(axis=1), 1.0)
    assert set(np.unique(model.predict(X))) <= {0, 1}

def test_output_contract(model, X):
    proba = model.predict_proba(X)
    assert proba.shape == (len(X), 2)
    assert np.all((proba >= 0) & (proba <= 1))
    assert np.allclose(proba.sum(axis=1), 1.0)
    assert set(np.unique(model.predict(X))) <= {0, 1}

Caught 0 of 6. Every broken model above produced perfectly valid probabilities. This test is still worth having — it catches the day someone returns log-odds instead of probabilities, or a column order flip in a multiclass output — but it is a contract, not a bug detector, and writing it should not feel like having tested the model.

2. Invariance tests

def test_row_order_invariance(model, X):
    """Predictions must not depend on the order rows arrive in."""
    base = model.predict_proba(X)[:, 1]
    order = np.random.default_rng(7).permutation(len(X))
    assert np.allclose(base[order], model.predict_proba(X.iloc[order])[:, 1])
 
 
def test_column_order_safety(model, X):
    """Reordering columns must either raise or produce identical predictions."""
    columns = list(X.columns)
    columns[0], columns[7] = columns[7], columns[0]
    try:
        other = model.predict_proba(X[columns])[:, 1]
    except Exception:
        return                                   # raising is the correct behaviour
    assert np.allclose(model.predict_proba(X)[:, 1], other)

def test_row_order_invariance(model, X):
    """Predictions must not depend on the order rows arrive in."""
    base = model.predict_proba(X)[:, 1]
    order = np.random.default_rng(7).permutation(len(X))
    assert np.allclose(base[order], model.predict_proba(X.iloc[order])[:, 1])
 
 
def test_column_order_safety(model, X):
    """Reordering columns must either raise or produce identical predictions."""
    columns = list(X.columns)
    columns[0], columns[7] = columns[7], columns[0]
    try:
        other = model.predict_proba(X[columns])[:, 1]
    except Exception:
        return                                   # raising is the correct behaviour
    assert np.allclose(model.predict_proba(X)[:, 1], other)

Row-order invariance caught 0 of 6. Column-order safety caught 1 — the serving path that converted to numpy and therefore indexed positionally, which is exactly the failure from the previous page. Note the shape of that assertion: raising is a pass. A test that demands no exception would have rewarded the buggy implementation.

3. The metric floor

def test_beats_floor(seed=0, floor=0.75):
    """A fixed seed, a fixed split, and a number the model must clear."""
    model = train(X_train, y_train)
    assert accuracy_score(y_test, model.predict(X_test)) >= floor

def test_beats_floor(seed=0, floor=0.75):
    """A fixed seed, a fixed split, and a number the model must clear."""
    model = train(X_train, y_train)
    assert accuracy_score(y_test, model.predict(X_test)) >= floor

Caught 5 of 6 — everything except the leaky scaler. It is the single most valuable test on the list and also the least diagnostic: a failure tells you the pipeline is broken and nothing about where. Two practical notes:

• Set the floor from a baseline, not from your best run. “Beats the majority class by 15 points” is a floor that survives; “at least 0.9187” is a floor that fails every time the seed changes.
• It cannot catch a bug that makes the metric better. Leakage is the whole family of such bugs, which is why the structural test below exists.

4. Memorise a tiny batch

def test_can_memorise_small_batch(n=24):
    """A correct training path fits 24 rows almost perfectly. A broken one cannot."""
    small_X, small_y = X_train.iloc[:n], y_train[:n]
    model = train(small_X, small_y)
    assert accuracy_score(small_y, model.predict(small_X)) >= 0.95

def test_can_memorise_small_batch(n=24):
    """A correct training path fits 24 rows almost perfectly. A broken one cannot."""
    small_X, small_y = X_train.iloc[:n], y_train[:n]
    model = train(small_X, small_y)
    assert accuracy_score(small_y, model.predict(small_X)) >= 0.95

Caught 4 of 6, in 5.6 ms, and it is the only test that points at the training loop specifically: shuffled labels, permuted rows, a single class and a missing transform all make memorisation impossible. This is the ML equivalent of a smoke test, it is standard practice in deep learning (“overfit one batch first”), and it is under-used everywhere else.

5. Directional expectations

def test_directional_expectation(model, X, feature, sign):
    """Pushing a feature in its known direction must raise the predicted probability."""
    base = model.predict_proba(X)[:, 1]
    nudged = X.copy()
    nudged[feature] = nudged[feature] + sign * 2 * X[feature].std()
    assert np.median(model.predict_proba(nudged)[:, 1] - base) > 0.01

def test_directional_expectation(model, X, feature, sign):
    """Pushing a feature in its known direction must raise the predicted probability."""
    base = model.predict_proba(X)[:, 1]
    nudged = X.copy()
    nudged[feature] = nudged[feature] + sign * 2 * X[feature].std()
    assert np.median(model.predict_proba(nudged)[:, 1] - base) > 0.01

Caught 4 of 6. This is the test that encodes domain knowledge: more income raises approval, more failed logins raises fraud risk, more tenure lowers churn. Two cautions:

• Assert a median increase, not “no decrease”. The weaker version passed for the shuffled-label model, whose coefficients were noise, because noise does not systematically decrease anything either.
• A violated directional expectation is not automatically a bug — it may be a real interaction, as on the PDP page. Treat a failure as a question, not a verdict.

6. The structural test

def test_all_preprocessing_inside_the_pipeline():
    """The returned object must be a Pipeline, so CV cannot leak through a transform."""
    model = train(X_train, y_train)
    assert isinstance(model, Pipeline)
    assert not isinstance(model[-1], TransformerMixin)      # last step predicts

def test_all_preprocessing_inside_the_pipeline():
    """The returned object must be a Pipeline, so CV cannot leak through a transform."""
    model = train(X_train, y_train)
    assert isinstance(model, Pipeline)
    assert not isinstance(model[-1], TransformerMixin)      # last step predicts

Caught 4 of 6, including the leaky scaler that no behavioural test could see. This is the only test in the set that checks how the model was built rather than what it does, and it is the answer to the class of bug that improves your metrics. It generalises:

• assert that every transformer is a pipeline step, so cross_val_scorecross_val_score refits it per fold
• assert that the fitted feature names match the schema artefact
• assert that no code path calls fitfit on data outside the training fold

Structural tests feel like cheating because they inspect the code rather than the results. They are the only defence against a bug whose symptom is a better number.

Arranging them so CI stays usable

  flowchart TD
  A["every commit
seconds"] --> B["output contract
invariances
column-order safety
structural assertions"]
  B --> C["every commit
under a minute"]
  C --> D["memorise 24 rows
directional expectations
feature parity on 500 ids"]
  D --> E["nightly, or on
release branches"]
  E --> F["metric floor on a fixed
seed and split
paired comparison against
the current production model"]
  F --> G["before promotion
manual"]
  G --> H["read the top coefficients
check the group metrics
re-derive the threshold
from the current costs"]

Two rules make the difference between a suite people keep and one they delete:

Fast tests must not need a trained model. Contract, invariance, column-order and structural tests can all run against a model fitted on 24 rows, or a stub. If your test suite trains a real model, it will be moved to nightly, and then it will not run.

One test per bug class, not per metric. Seven assertions that all fail when accuracy drops are one test. The suite above is small because each entry catches something the others cannot: the matrix has no duplicate columns.

See it move

The matrix is a coverage problem, so it can be played with directly. Click a test to add or remove it from the suite; the sketch shows which of the six bugs the suite catches, which slip through, and what the suite costs. Every cell is the measured matrix from the figure above, not an estimate.

Two results fall out of the matrix as soon as you can toggle it. The cheapest suite that catches all six is metric floormetric floor + everything in a Pipelineeverything in a Pipeline, 20.6 ms — five bugs from the floor and the leaky scaler from the structural check. And everything in a Pipelineeverything in a Pipeline is irreplaceable: turn it off and the leaky scaler escapes every remaining test, however many you enable, because its symptom is a better number. The other five tests are redundant on this particular bug set — which is not an argument for deleting them, since each exists to catch a bug class this experiment did not inject, but it is an argument for knowing which of your tests are load-bearing and which are habits.

Pitfalls

Recap

• Seven realistic bugs; accuracy from 0.8500 (leaky scaler, indistinguishable from correct) down to 0.4592.
• The output contract and row-order invariance tests caught 0 of 6.
• The metric floor caught 5 of 6 and localises nothing.
• Memorising 24 rows caught 4 of 6 in 5.6 ms and points straight at the training loop.
• Directional expectations caught 4 of 6 — with a median-increase assertion, not “no decrease”.
• The structural test was the only one that caught the leaky scaler, because that bug does not change any metric.
• Column-order safety caught the positional serving path, and only because raising counts as a pass.

🧪 Try It Yourself

Exercise 1 – Build the correct pipeline and two bugs

Exercise 2 – The tests that catch nothing

Exercise 3 – Memorise twenty-four rows

Exercise 4 – Find the batch size where the test breaks

Exercise 5 – The structural test

Next

Phase 11 - ML Engineering — the phase overview, with the four measurements that decide whether a model survives contact with a production codebase.