Multiprocessing in Python

What is multiprocessing?

Multiprocessing runs work in multiple processes.

• Each process has its own Python interpreter and memory space.
• This can use multiple CPU cores.

Why multiprocessing helps CPU-heavy work

In CPython, threads are limited by the GIL for CPU-bound code.

Multiprocessing avoids this because:

• each process has its own GIL

When to use multiprocessing

Best for CPU-bound tasks:

• heavy number crunching
• image processing
• data transformations
• simulations

For I/O-bound tasks, threading or async may be better.

Important note (Windows / notebooks)

When using multiprocessing, always guard code with:

if __name__ == "__main__":
    # start processes here
    pass

This avoids infinite child-process spawning on some platforms.

Visualize it

Unlike threads, which share one interpreter and memory space, each process gets its own — so they run truly in parallel on separate CPU cores with no GIL to fight over:

  flowchart TB
  Parent[Parent process]
  Parent --> P1[Process 1
own memory + own interpreter]
  Parent --> P2[Process 2
own memory + own interpreter]
  Parent --> P3[Process 3
own memory + own interpreter]
  P1 --- C1[CPU core 1]
  P2 --- C2[CPU core 2]
  P3 --- C3[CPU core 3]
  P1 -. pickled data via Queue/Pipe .-> Parent
  P2 -. pickled data via Queue/Pipe .-> Parent
  P3 -. pickled data via Queue/Pipe .-> Parent

🧪 Try It Yourself

Exercise 1 – Start a Process

Press Run to see what your code prints.

Exercise 2 – Process Pool map()

Press Run to see what your code prints.

Exercise 3 – Multiprocessing Queue

Press Run to see what your code prints.