Organizing Tests into Test Suites

Why test suites?

Suites let you:

• group tests by feature
• run a subset for smoke/regression
• create custom runners

Example suite

suite_example.py

import unittest
 
 
class TestA(unittest.TestCase):
    def test_one(self):
        self.assertTrue(True)
 
 
class TestB(unittest.TestCase):
    def test_two(self):
        self.assertEqual(1 + 1, 2)
 
 
def load_suite():
    suite = unittest.TestSuite()
    suite.addTest(unittest.defaultTestLoader.loadTestsFromTestCase(TestA))
    suite.addTest(unittest.defaultTestLoader.loadTestsFromTestCase(TestB))
    return suite
 
 
if __name__ == "__main__":
    runner = unittest.TextTestRunner(verbosity=2)
    runner.run(load_suite())

Tip

In most modern codebases, you’ll rely on discovery.

Suites are most useful for curated runs.

A TestSuite is a container you build yourself

Discovery is the usual entry point. A TestSuite is what you reach for when you need to choose and order the tests explicitly:

suite.py

import unittest
from tests.test_calc import TestAdd
from tests.test_msgs import TestMessages
 
def build_suite():
    suite = unittest.TestSuite()
    loader = unittest.TestLoader()
 
    suite.addTest(loader.loadTestsFromTestCase(TestAdd))
    suite.addTest(TestMessages("test_in"))        # one specific method
    return suite
 
if __name__ == "__main__":
    runner = unittest.TextTestRunner(verbosity=2)
    result = runner.run(build_suite())
    raise SystemExit(0 if result.wasSuccessful() else 1)

diagram Diagram mermaid

  flowchart TD
  L["TestLoader"] --> A["loadTestsFromTestCase(Class)"]
  L --> B["loadTestsFromModule(module)"]
  L --> C["loadTestsFromName('pkg.mod.Class.method')"]
  A --> S["TestSuite"]
  B --> S
  C --> S
  S --> N["suites can contain suites"]
  S --> R["TextTestRunner(verbosity=2).run(suite)"]
  R --> W["result.wasSuccessful() -> your exit code"]

raise SystemExit(...) on the last line is the part people leave out. Without it, a custom runner script exits 0 whatever happened, and CI reports success on a failing suite.

Grouping by speed rather than by module

The useful grouping is usually not “all the model tests” but “the tests I can afford to run on every save”:

grouped.py

def fast_suite():
    loader = unittest.TestLoader()
    suite = unittest.TestSuite()
    for cls in (TestAdd, TestMessages):
        for name in loader.getTestCaseNames(cls):
            if not name.endswith("_slow"):
                suite.addTest(cls(name))
    return suite

This is exactly what pytest markers do declaratively, which is one of the reasons the next phase exists.

Sharing expensive setup across a whole suite

shared.py

def setUpModule():
    global _server
    _server = start_test_server()      # once for the entire module
 
def tearDownModule():
    _server.stop()

hook	runs
setUpModule	once per module
setUpClass	once per class
setUp	before every test method

Push shared setup as far up as it will go only when it is genuinely read-only. A database connection shared across a class is fine; a mutable object shared across tests reintroduces exactly the coupling setUp exists to prevent.

Shared state is how order-dependence gets in

Since tests run alphabetically, a suite that passes today can fail after a rename if one test mutates something another expects. If you suspect it, run a single test on its own — if it fails alone and passes in the suite, the state is shared.

See it move

sketch Building a suite from parts p5.js Pause Restart

A loader turns classes and names into tests, a suite holds them in the order you add them, and the runner's result decides your exit code.

Check yourself

pch.quizTag pch.quizDefaultTitle

1. You write a script that builds a TestSuite and runs it with TextTestRunner. What must the script do that discovery does for you?

   A import unittest B set the exit code from result.wasSuccessful(), or CI reports success on a failing suite C call setUp manually D sort the tests
   pch.quizShowAnswer

   B — set the exit code from result.wasSuccessful(), or CI reports success on a failing suite — A custom runner script exits 0 by default whatever the result. raise SystemExit(0 if result.wasSuccessful() else 1) is the missing line.

2. Which hook runs once per module?

   A setUp B setUpClass C setUpModule D tearDown
   pch.quizShowAnswer

   C — setUpModule — setUpModule runs once per module, setUpClass once per class, and setUp before every test method. Push shared setup upward only when it is read-only.

3. A test passes in the suite but fails when run alone. What does that indicate?

   A a bug in unittest B the test depends on state left behind by another test C the test is too slow D discovery is misconfigured
   pch.quizShowAnswer

   B — the test depends on state left behind by another test — Since tests run alphabetically, shared mutable state creates order-dependence that survives until someone renames a method.