Python Concurrency — threading, multiprocessing, asyncio

Concurrency means making progress on more than one task at a time. Python offers three approaches, each suited to a different kind of work:

Approach	Best for	Why
threading	I/O-bound work (network, disk)	Threads wait together; cheap to create.
multiprocessing	CPU-bound work (number crunching)	Separate processes use multiple cores.
asyncio	Many I/O tasks at once	One thread, cooperative await, very scalable.

The GIL — why it matters

CPython has a Global Interpreter Lock (GIL): only one thread executes Python bytecode at a time. So threads don’t speed up CPU-bound code — but they’re great for I/O, where threads spend most time waiting (and release the GIL while waiting). For CPU-bound parallelism, use multiprocessing, which sidesteps the GIL with separate processes.

Rule of thumb: I/O-bound → threading or asyncio. CPU-bound → multiprocessing.

threading

Run functions in separate threads. Start them, then join to wait for completion.

threading_basic.py

import threading
import time
 
def worker(name):
    print(f"{name} starting")
    time.sleep(1)               # simulates an I/O wait
    print(f"{name} done")
 
threads = [threading.Thread(target=worker, args=(f"T{i}",)) for i in range(3)]
for t in threads:
    t.start()                   # begin running concurrently
for t in threads:
    t.join()                    # wait for each to finish
print("all done")

Because the threads sleep concurrently, three 1-second waits finish in about 1 second, not 3.

Sharing data safely with a Lock

When threads modify shared state, guard it with a Lock to avoid race conditions.

lock.py

import threading
 
counter = 0
lock = threading.Lock()
 
def increment():
    global counter
    for _ in range(100_000):
        with lock:              # only one thread at a time
            counter += 1
 
ts = [threading.Thread(target=increment) for _ in range(2)]
for t in ts: t.start()
for t in ts: t.join()
print(counter)                  # 200000 (correct, thanks to the lock)

concurrent.futures — a simpler interface

ThreadPoolExecutor / ProcessPoolExecutor manage a pool of workers and collect results for you.

executor.py

from concurrent.futures import ThreadPoolExecutor
 
def square(n):
    return n * n
 
with ThreadPoolExecutor(max_workers=4) as pool:
    results = list(pool.map(square, [1, 2, 3, 4, 5]))
print(results)                  # [1, 4, 9, 16, 25]

Swap ThreadPoolExecutor for ProcessPoolExecutor to run CPU-bound work across cores — the API is identical.

multiprocessing

Each process has its own Python interpreter and memory, so they run truly in parallel on multiple cores.

multiprocessing_basic.py

from multiprocessing import Pool
 
def heavy(n):
    return sum(i * i for i in range(n))
 
if __name__ == "__main__":          # required guard on Windows/macOS
    with Pool(processes=4) as pool:
        results = pool.map(heavy, [10_000, 20_000, 30_000])
    print(results)

Always wrap multiprocessing entry code in if __name__ == "__main__": — without it, spawned processes re-import and re-run your script.

asyncio — async/await

asyncio runs many I/O tasks on a single thread using cooperative multitasking. An async def function is a coroutine; await yields control while waiting, letting other coroutines run.

asyncio_basic.py

import asyncio
 
async def fetch(name, delay):
    print(f"{name} start")
    await asyncio.sleep(delay)      # non-blocking wait
    print(f"{name} done")
    return name
 
async def main():
    # Run three coroutines concurrently
    results = await asyncio.gather(
        fetch("A", 1),
        fetch("B", 1),
        fetch("C", 1),
    )
    print(results)
 
asyncio.run(main())                 # entry point

All three “fetches” overlap, so the total time is about 1 second, not 3.

Key asyncio pieces

Piece	Purpose
async def	Defines a coroutine.
await x	Wait for an awaitable without blocking the thread.
asyncio.run(coro)	Run the top-level coroutine.
asyncio.gather(*coros)	Run many coroutines concurrently, collect results.
asyncio.create_task(coro)	Schedule a coroutine to run in the background.
asyncio.sleep(s)	Non-blocking sleep.

Choosing the right tool

decision.txt

Is the work CPU-bound (math, parsing, compression)?
   -> multiprocessing (use all cores, beat the GIL)
 
Is the work I/O-bound (HTTP, files, DB)?
   A few tasks, simple code?      -> threading / ThreadPoolExecutor
   Hundreds/thousands of tasks?   -> asyncio

Common pitfalls

• Threads won’t speed up CPU work — the GIL serializes them; use multiprocessing.
• Forgetting join — the main program may exit before threads finish.
• Unguarded shared state — use a Lock around shared mutable data.
• Blocking calls inside asyncio — time.sleep or heavy CPU work freezes the whole event loop; use await asyncio.sleep and run CPU work in an executor.
• Missing if __name__ == "__main__": — breaks multiprocessing on Windows/macOS.

Practice Exercises

Exercise 1 – Run work in threads

pch.exerciseTag pch.exerciseTitle

Exercise 2 – Define and run a coroutine

pch.exerciseTag pch.exerciseTitle

Exercise 3 – Map work over a thread pool

pch.exerciseTag pch.exerciseTitle

Choosing between the three

The choice is not a matter of taste. It follows from one question — is your program waiting, or is it computing?

diagram Diagram mermaid

  flowchart TD
  S["your task is slow"] --> Q{"slow because it..."}
  Q -->|"waits on network, disk, a subprocess"| IO["I/O-bound"]
  Q -->|"burns CPU: parsing, math, compression"| CPU["CPU-bound"]
  IO --> Q2{"how many waits at once?"}
  Q2 -->|"dozens"| TH["threading
simplest, blocking libraries work unchanged"]
  Q2 -->|"thousands"| AS["asyncio
far cheaper per task, needs async libraries"]
  CPU --> MP["multiprocessing
the only one that uses more than one core"]

The dividing line is the GIL: one lock that lets only one thread execute Python bytecode at a time. A thread that is waiting releases it, so waiting overlaps freely. A thread that is computing holds it, so computing does not overlap at all.

The measurement that shows the GIL

Wall-clock speedup on a shared machine is noisy. A ratio measured inside one run is not. Time the CPU each worker actually consumed and divide by the elapsed time:

overlap.py

import time
from concurrent.futures import ThreadPoolExecutor, ProcessPoolExecutor
 
def cpu_thread(n):                       # per-THREAD cpu time
    t = time.thread_time()
    s = 0
    for i in range(n):
        s += i * i
    return time.thread_time() - t
 
def cpu_proc(n):                         # per-PROCESS cpu time
    t = time.process_time()
    s = 0
    for i in range(n):
        s += i * i
    return time.process_time() - t
 
if __name__ == "__main__":               # required on Windows and macOS
    N = 10_000_000
    t = time.perf_counter()
    with ThreadPoolExecutor(4) as ex:
        d = list(ex.map(cpu_thread, [N] * 4))
    print(sum(d) / (time.perf_counter() - t))    # 1.00
 
    t = time.perf_counter()
    with ProcessPoolExecutor(4) as ex:
        d = list(ex.map(cpu_proc, [N] * 4))
    print(sum(d) / (time.perf_counter() - t))    # 3.05

Measured on CPython 3.14.4, 8 logical CPUs, four workers each squaring 10 million integers:

pool	wall	CPU consumed	CPU ÷ wall	reading
4 threads	2.72 s	2.72 s	1.00×	perfectly serialized
4 processes	1.53 s	4.67 s	3.05×	genuinely parallel

The threads consumed exactly as much CPU as the clock advanced: at any instant, one of them was running. The processes consumed 4.67 s of CPU in 1.53 s of wall time, which is only possible on more than one core.

Measure CPU time, not elapsed time

Timing a worker with time.perf_counter measures how long it was alive, including time parked waiting for the GIL. All four threads span the whole window, so that metric reports near-perfect “overlap” and appears to disprove the GIL. time.thread_time and time.process_time count CPU actually burned, which is the question being asked.

See it move

One GIL, four threads. Only the thread holding the baton runs; the others queue. Switch to processes and each gets its own interpreter — and its own baton. Toggle the workload to see why waiting changes everything.

sketch The GIL as a baton p5.js Pause Restart

CPU-bound threads must pass one baton so only one runs at a time. Processes each hold their own. Waiting threads release the baton, so I/O overlaps.

Waiting is where threads shine

Ten tasks that each sleep 0.1 s, measured on the same machine:

approach	wall	speedup
sequential	1.005 s	1.00×
ThreadPoolExecutor(10)	0.110 s	9.17×
asyncio.gather of 10	0.116 s	8.64×

time.sleep releases the GIL, and so does every well-behaved socket and file call. Ten threads therefore wait simultaneously and the total is set by the longest wait rather than their sum.

Threads and coroutines are equally fast here. What separates them is cost per task:

cost.py

# 500 threads that do nothing   -> 104.2 ms   (208 us each)
# 500 coroutines that do nothing ->  3.0 ms   (6 us each)

Roughly 35× cheaper. A thread needs an OS stack; a coroutine is an object. At ten concurrent waits the difference is irrelevant — at ten thousand, it decides whether the program runs at all.

Processes are not free

Starting a 4-worker pool and running four trivial tasks took about 1 second on this machine, and every argument and result must be pickled and copied between processes. For work measured in milliseconds a process pool is reliably slower than doing it sequentially. It pays off only when each task runs long enough to dwarf that setup.

The __main__ guard is mandatory

On Windows and macOS, multiprocessing starts workers with spawn: each child re-imports your module. Without if __name__ == "__main__": around the code that creates the pool, every child creates another pool. Symptoms range from RuntimeError to an endless fork bomb.

Check yourself

pch.quizTag pch.quizDefaultTitle

1. Four CPU-bound tasks run in a ThreadPoolExecutor(4). Measured CPU time divided by wall time is 1.00. What does that mean?

   A the threads used four cores efficiently B only one thread executed Python bytecode at a time, so nothing overlapped C the measurement failed, since the ratio can never be 1 D the tasks were actually I/O-bound
   pch.quizShowAnswer

   B — only one thread executed Python bytecode at a time, so nothing overlapped — A ratio of 1 means the workers together burned exactly one core-second per second of clock. That is the GIL serializing them. The same measurement over four processes gives 3.05.

2. Why measure workers with time.thread_time rather than time.perf_counter?

   A perf_counter has too little resolution for short tasks B perf_counter measures elapsed time, so a thread blocked on the GIL still accrues duration C thread_time is the only clock available inside a thread D they are interchangeable, thread_time is just shorter to type
   pch.quizShowAnswer

   B — perf_counter measures elapsed time, so a thread blocked on the GIL still accrues duration — All four threads are alive for the whole window, so summing their elapsed times reports near-perfect overlap and appears to disprove the GIL. thread_time counts CPU actually consumed.

3. For ten tasks that each sleep 0.1 s, threads gave 9.17x and asyncio 8.64x. What separates them?

   A nothing measurable here; coroutines cost about 35x less to create, which matters at large scale B asyncio is faster because it avoids the GIL entirely C threads are faster because the OS schedules them on multiple cores D asyncio is slower because coroutines cannot overlap waits
   pch.quizShowAnswer

   A — nothing measurable here; coroutines cost about 35x less to create, which matters at large scale — Both overlap the waiting, so both approach 10x. Measured creation cost was 208 us per thread against 6 us per coroutine, which decides things at thousands of concurrent tasks, not ten.

4. When is a process pool likely to be SLOWER than a plain sequential loop?

   A when the tasks are CPU-bound B when there are more cores than tasks C when each task is short, because pool startup and pickling dominate D never; processes are always at least as fast
   pch.quizShowAnswer

   C — when each task is short, because pool startup and pickling dominate — Starting a 4-worker pool cost about 1 second here, plus pickling every argument and result. Tasks measured in milliseconds cannot repay that.

Summary

• The GIL means threads don’t parallelize CPU work — they shine for I/O.
• threading + Lock for simple concurrent I/O; concurrent.futures for pools and results.
• multiprocessing for CPU-bound parallelism across cores (guard with __main__).
• asyncio (async/await, gather, run) scales to many concurrent I/O tasks on one thread.
• Match the tool to the workload: CPU → processes, I/O → threads/async.