Generating HTML Test Reports

Why HTML reports

Useful for:

• sharing results with teams
• CI artifacts
• historical tracking

Common plugin

• pytest-html

Example usage

bash

pytest --html=report.html --self-contained-html

Tips

• store reports as CI artifacts
• include environment info and versions

Two different reports, often confused

commands

pytest --html=report.html --self-contained-html       # which TESTS ran
pytest --cov=shop --cov-report=html                    # which CODE ran

diagram Diagram mermaid

  flowchart TD
  P["pytest-html"] --> A["report.html
every test, outcome, duration, captured output"]
  C["coverage html"] --> B["htmlcov/index.html
per-file source with covered lines marked"]
  A --> Q1["answers: what failed, and what did it print?"]
  B --> Q2["answers: which lines did the tests never reach?"]

Measured artefacts from one run:

file	size	contents
report.html	37,274 bytes	the test run, self-contained
htmlcov/index.html	4,686 bytes	per-file coverage, plus assets
coverage.xml	1,279 bytes	line-rate="0.8333" for CI tools
.coverage	53,248 bytes	the raw SQLite data file

--self-contained-html inlines the CSS and JavaScript so the file survives being emailed or stored as a CI artefact. Without it the report needs its assets/ directory alongside, and it arrives unstyled.

Machine-readable output matters more than the HTML

for CI

pytest --junitxml=results.xml        # test results, understood by most CI systems
pytest --cov --cov-report=xml        # coverage.xml for Codecov, SonarQube and friends

Measured coverage.xml carried line-rate="0.8333", matching the 83% in the terminal. CI systems parse these to annotate a pull request with which tests failed and which lines a change left uncovered — which is where the information is actually useful, rather than in a file someone has to open.

Publishing them

.github/workflows/tests.yml

- name: Run tests
  run: pytest --cov=myapp --cov-report=xml --junitxml=results.xml --html=report.html --self-contained-html
 
- name: Upload reports
  if: always()          # otherwise a failing test suite uploads nothing
  uses: actions/upload-artifact@v4
  with:
    name: test-reports
    path: |
      report.html
      results.xml
      coverage.xml

if: always() is the line worth copying. Without it the upload step is skipped when tests fail — which is precisely the run whose report you wanted.

Keep them out of version control

.gitignore

report.html
htmlcov/
coverage.xml
.coverage
.pytest_cache/

These are build outputs. Committing them produces enormous diffs on every run and they are stale the moment anything changes.

A report is a record, not a signal

Nobody opens an HTML report on a green build. The number that changes behaviour is the exit code and the pull request annotation; the report is for the run where you already know something is wrong and need the detail.

See it move

sketch Which report answers which question p5.js Pause Restart

pytest-html records the test run. coverage html records which lines the run reached. They are complementary, not alternatives.

Check yourself

pch.quizTag pch.quizDefaultTitle

1. What is the difference between report.html from pytest-html and htmlcov/index.html from coverage?

   A they are the same report in different formats B one records which tests ran and what they printed; the other records which lines of source the run reached C htmlcov is a summary of report.html D report.html only appears when tests fail
   pch.quizShowAnswer

   B — one records which tests ran and what they printed; the other records which lines of source the run reached — They answer different questions and are complementary. Measured 37,274 bytes for the test report and 4,686 for the coverage index.

2. Why pass --self-contained-html?

   A it makes the report smaller B it inlines the CSS and JavaScript so the single file survives being emailed or stored as a CI artefact C it is required for coverage D it removes captured output
   pch.quizShowAnswer

   B — it inlines the CSS and JavaScript so the single file survives being emailed or stored as a CI artefact — Without it the report needs its assets directory alongside and arrives unstyled.

3. Why does the artifact-upload step in CI need if: always()?

   A to upload faster B without it the step is skipped when tests fail — exactly the run whose report you wanted C it is required by the action D to overwrite the previous artifact
   pch.quizShowAnswer

   B — without it the step is skipped when tests fail — exactly the run whose report you wanted — Steps are skipped by default once a previous step fails, so the report from the interesting run is the one you would lose.