Skip to content

Process Pool (Pool, map, starmap)

Why use a process pool

Creating many processes manually is not ideal.

A pool:

  • reuses worker processes
  • makes parallel execution easier

Pool.map

pool_map.py
from multiprocessing import Pool
 
 
def square(x: int) -> int:
    return x * x
 
 
if __name__ == "__main__":
    with Pool(processes=4) as pool:
        results = pool.map(square, [1, 2, 3, 4, 5])
 
    print(results)
pool_map.py
from multiprocessing import Pool
 
 
def square(x: int) -> int:
    return x * x
 
 
if __name__ == "__main__":
    with Pool(processes=4) as pool:
        results = pool.map(square, [1, 2, 3, 4, 5])
 
    print(results)

Pool.starmap (multiple arguments)

pool_starmap.py
from multiprocessing import Pool
 
 
def power(x: int, p: int) -> int:
    return x ** p
 
 
if __name__ == "__main__":
    with Pool(processes=2) as pool:
        results = pool.starmap(power, [(2, 3), (3, 2), (4, 2)])
 
    print(results)
pool_starmap.py
from multiprocessing import Pool
 
 
def power(x: int, p: int) -> int:
    return x ** p
 
 
if __name__ == "__main__":
    with Pool(processes=2) as pool:
        results = pool.starmap(power, [(2, 3), (3, 2), (4, 2)])
 
    print(results)

Common mistakes

  • Forgetting the if __name__ == "__main__"if __name__ == "__main__" guard.
  • Passing non-picklable objects (like lambdas on some platforms).

๐Ÿงช Try It Yourself

Exercise 1 โ€“ Start a Process

Exercise 2 โ€“ Process Pool map()

Exercise 3 โ€“ Multiprocessing Queue

If this helped you, consider buying me a coffee โ˜•

Buy me a coffee

Was this page helpful?

Let us know how we did