Thread Pool with concurrent.futures

Why use ThreadPoolExecutor

Creating threads manually is okay for a few tasks.

For many tasks, prefer:

concurrent.futures.ThreadPoolExecutor

It provides:

a thread pool
easy result collection
simpler error handling

Example: run many tasks

thread_pool.py

from concurrent.futures import ThreadPoolExecutor, as_completed
import time
 
 
def fetch(i: int) -> int:
    time.sleep(0.2)
    return i * i
 
 
results = []
with ThreadPoolExecutor(max_workers=5) as ex:
    futures = [ex.submit(fetch, i) for i in range(10)]
 
    for f in as_completed(futures):
        results.append(f.result())
 
print(sorted(results))

Exception handling

If a worker raises an exception, future.result() raises it.

thread_pool_errors.py

from concurrent.futures import ThreadPoolExecutor
 
 
def work(x: int) -> float:
    return 10 / x
 
with ThreadPoolExecutor() as ex:
    futures = [ex.submit(work, x) for x in [2, 1, 0]]
 
    for f in futures:
        try:
            print(f.result())
        except Exception as e:
            print("Worker failed:", e)

sketch Threads are for waiting: 9x on I/O, and nothing on CPU p5.js

Twenty tasks that each sleep for 50 milliseconds -- a stand-in for a network call or a disk read. Sequentially that is a full second of doing nothing. A thread pool overlaps the waiting, and the speedup tracks the worker count almost exactly: four workers gave 3.9x and ten gave 9.2x. The same pool on CPU-bound work measured 0.92x -- slower than not using it at all -- because the GIL lets only one thread run Python bytecode. The task decides, not the pool.

Thread Pool with concurrent.futures

Why use ThreadPoolExecutor

Example: run many tasks

Exception handling

When to prefer a thread pool

🧪 Try It Yourself

Exercise 1 – Submit a Task

Exercise 2 – Map Over a List

Exercise 3 – Check Future Done

pch.feedbackHeading