Process Pool (Pool, map, starmap)
Why use a process pool
Creating many processes manually is not ideal.
A pool:
- reuses worker processes
- makes parallel execution easier
Pool.map
pool_map.py
from multiprocessing import Pool
def square(x: int) -> int:
return x * x
if __name__ == "__main__":
with Pool(processes=4) as pool:
results = pool.map(square, [1, 2, 3, 4, 5])
print(results)pool_map.py
from multiprocessing import Pool
def square(x: int) -> int:
return x * x
if __name__ == "__main__":
with Pool(processes=4) as pool:
results = pool.map(square, [1, 2, 3, 4, 5])
print(results)Pool.starmap (multiple arguments)
pool_starmap.py
from multiprocessing import Pool
def power(x: int, p: int) -> int:
return x ** p
if __name__ == "__main__":
with Pool(processes=2) as pool:
results = pool.starmap(power, [(2, 3), (3, 2), (4, 2)])
print(results)pool_starmap.py
from multiprocessing import Pool
def power(x: int, p: int) -> int:
return x ** p
if __name__ == "__main__":
with Pool(processes=2) as pool:
results = pool.starmap(power, [(2, 3), (3, 2), (4, 2)])
print(results)Common mistakes
- Forgetting the
if __name__ == "__main__"if __name__ == "__main__"guard. - Passing non-picklable objects (like lambdas on some platforms).
๐งช Try It Yourself
Exercise 1 โ Start a Process
Exercise 2 โ Process Pool map()
Exercise 3 โ Multiprocessing Queue
If this helped you, consider buying me a coffee โ
Buy me a coffeeWas this page helpful?
Let us know how we did
