Measuring Code Coverage with coverage.py

What coverage tells you

Coverage measures which lines/branches ran during tests.

Coverage helps you find:

• untested code paths
• dead code

Coverage does not guarantee correctness.

Using pytest-cov

Common setup:

• pytest-cov uses coverage.py

Run:

bash

pytest --cov=src --cov-report=term-missing

HTML report:

bash

pytest --cov=src --cov-report=html

Good practice

• aim for meaningful coverage
• focus on risk areas
• don’t chase 100% blindly

Running it

commands

pytest --cov=shop --cov-report=term-missing
pytest --cov=shop --cov-report=html         # browsable, htmlcov/index.html
pytest --cov=shop --cov-report=xml          # for CI tools
pytest --cov=shop --cov-fail-under=80       # exit 1 below the threshold

Measured on a small module:

term-missing

Name      Stmts   Miss  Cover   Missing
---------------------------------------
shop.py      12      2    83%   11, 14

Missing is the column that matters. 83% tells you how much; lines 11 and 14 tell you what to do next.

The measurement that should change how you read a coverage number

A module with one if, and a single test that only exercises the true branch:

branchy.py

def classify(n):
    label = "small"
    if n > 10:
        label = "big"
    return label

test_branchy.py

def test_big():
    assert classify(50) == "big"

statement coverage

branchy.py   5   0   100%

with --cov-branch

branchy.py   5   0   2   1   86%   3->5

100% against 86%. Nothing ever passed a small number, so the if was never false, and the 3->5 branch — falling straight from the condition to the return — was never taken. Statement coverage cannot see it, because every line did execute.

diagram Diagram mermaid

  flowchart TD
  S["statement coverage"] --> Q1["did this LINE run?"]
  B["branch coverage"] --> Q2["did each OUTCOME of each condition happen?"]
  Q1 --> H["an if with no else can be 100% with one test"]
  Q2 --> D["3->5 reported as a missing branch"]

Turn it on. It is one flag:

pyproject.toml

[tool.coverage.run]
branch = true
source = ["myapp"]
omit = ["*/migrations/*", "*/tests/*"]
 
[tool.coverage.report]
show_missing = true
skip_covered = true
exclude_lines = [
  "pragma: no cover",
  "if TYPE_CHECKING:",
  "raise NotImplementedError",
]

Threshold behaviour

measured exit codes

pytest --cov=shop --cov-fail-under=90     # actual 83%  ->  exit 1
pytest --cov=shop                          #             ->  exit 0

A threshold is a ratchet, not a target. Setting it slightly below the current number stops coverage falling; setting it at 100 encourages tests written to touch lines rather than to check behaviour.

Coverage measures execution, not verification

A test that calls a function and asserts nothing gives full coverage of it. This passes, and covers every line:

worthless.py

def test_it_runs():
    price_with_tax(100)        # no assert

Coverage tells you what your tests touched. It cannot tell you what they checked.

What to do with the report

The useful reading is not the percentage but the Missing column and the HTML report’s red lines. Uncovered error handling and uncovered branches are where bugs live, because they are exactly the paths nobody exercised by hand either.

See it move

sketch 100% statement coverage, 86% branch coverage p5.js Pause Restart

One test through the true side of a condition executes every line while never taking the false branch.

Check yourself

pch.quizTag pch.quizDefaultTitle

1. A module scored 100% statement coverage and 86% branch coverage from the same single test. How?

   A the branch measurement is unreliable B every line executed, but one outcome of an if never happened — the false branch was never taken C branch coverage counts differently by design and is always lower D the test failed
   pch.quizShowAnswer

   B — every line executed, but one outcome of an if never happened — the false branch was never taken — Measured the missing branch reported as 3->5. An if with no else can reach 100% statement coverage from a single test that only takes the true side.

2. What does this test contribute to coverage: def test_it_runs(): price_with_tax(100)?

   A nothing, because it has no assert B full coverage of every line it executes, while verifying nothing at all C coverage only counts if an assertion passes D it fails the run
   pch.quizShowAnswer

   B — full coverage of every line it executes, while verifying nothing at all — Coverage measures execution, not verification. It tells you what your tests touched, never what they checked.

3. pytest --cov=shop --cov-fail-under=90 on a module at 83% exits with what code?

   A 0 B 1 C 5 D 83
   pch.quizShowAnswer

   B — 1 — Measured exit 1. A threshold works best as a ratchet just below the current number, to stop coverage falling rather than to chase a target.