Skip to content

Multiprocessing in Python

What is multiprocessing?

Multiprocessing runs work in multiple processes.

  • Each process has its own Python interpreter and memory space.
  • This can use multiple CPU cores.

Why multiprocessing helps CPU-heavy work

In CPython, threads are limited by the GIL for CPU-bound code.

Multiprocessing avoids this because:

  • each process has its own GIL

When to use multiprocessing

Best for CPU-bound tasks:

  • heavy number crunching
  • image processing
  • data transformations
  • simulations

For I/O-bound tasks, threading or async may be better.

Important note (Windows / notebooks)

When using multiprocessing, always guard code with:

main_guard.py
if __name__ == "__main__":
    # start processes here
    pass
main_guard.py
if __name__ == "__main__":
    # start processes here
    pass

This avoids infinite child-process spawning on some platforms.

๐Ÿงช Try It Yourself

Exercise 1 โ€“ Start a Process

Exercise 2 โ€“ Process Pool map()

Exercise 3 โ€“ Multiprocessing Queue

If this helped you, consider buying me a coffee โ˜•

Buy me a coffee

Was this page helpful?

Let us know how we did