Asyncio Synchronization (Lock, Semaphore)

diagram an async semaphore limits concurrency without blocking a thread mermaid

  flowchart TD
  A["async with sem:"] --> B{"a permit free?"}
  B -->|yes| C["take it, run the body"]
  B -->|no| D["SUSPEND -- the loop runs other tasks"]
  D --> E["a permit is released"]
  E --> C
  C --> F["body awaits I/O"]
  F --> G["leaving the block releases the permit"]
  G --> H["even if the body raised"]
  H --> I["a waiting task is resumed"]

There is one thread, so a task waiting for a permit must not block -- it suspends, and the loop runs something else. That is the difference from a threading semaphore: the ceiling is the same, but waiting is free. The permit is released by leaving the async with block, including when the body raises.

Measured: the ceiling is exact, the speedup is not quite linear

Twelve jobs, each await asyncio.sleep(0.05):

Limit	Peak in flight	Elapsed	Ideal
Semaphore(2)	2	0.37 s	0.30 s
Semaphore(4)	4	0.19 s	0.15 s
Semaphore(12)	12	0.06 s	0.05 s

The peak never exceeded the limit — the ceiling is a real guarantee, not an average. The elapsed times run slightly over the ideal because each wave has to be scheduled, and that overhead is a larger share of the total as the waves get shorter.

Use asyncio.Semaphore, not threading.Semaphore, in async code. The threading version blocks the whole thread when it waits, which in an event loop means blocking every task, including the ones holding the permits you are waiting for — a deadlock rather than a slowdown.

diagram an asyncio.Lock protects across awaits, which is the whole point mermaid

  flowchart TD
  A["task A: async with lock"] --> B["reads shared state"]
  B --> C["await -- control goes to the loop"]
  C --> D{"task B wants the lock"}
  D --> E["B suspends; it does not enter"]
  C --> F["A resumes, writes, leaves the block"]
  F --> G["lock released"]
  G --> H["B enters and runs the whole section"]
  I["no await inside the section?"] --> J["you do not need a lock at all"]

Ordinary async code needs no lock, because control only changes hands at an await you can see. A lock is needed exactly when a critical section CONTAINS an await -- read, await, write -- because another task can run in that gap. The lock makes the whole section atomic with respect to other tasks.

Why async locks exist

Even in a single-threaded event loop, tasks can interleave at await points.

So if a task:

• reads shared state
• awaits
• writes shared state

…another task can run in-between.

asyncio.Lock

async_lock.py

import asyncio
 
 
lock = asyncio.Lock()
count = 0
 
 
async def inc():
    global count
    for _ in range(10_000):
        async with lock:
            count += 1
 
 
async def main():
    await asyncio.gather(inc(), inc())
    print(count)
 
 
asyncio.run(main())

asyncio.Semaphore (limit concurrency)

async_semaphore.py

import asyncio
 
sem = asyncio.Semaphore(3)
 
 
async def worker(i: int):
    async with sem:
        print("start", i)
        await asyncio.sleep(0.2)
        print("end", i)
 
 
async def main():
    await asyncio.gather(*(worker(i) for i in range(10)))
 
 
asyncio.run(main())

sketch A semaphore caps how many coroutines are inside at once p5.js Pause Restart

Twelve jobs of 0.05 seconds each, run under three different permit counts. The ceiling was never exceeded in any run -- peak in flight was exactly the limit. What changes is how many waves the work takes, and the elapsed times run slightly over the ideal because every wave has to be scheduled.

Check yourself

pch.quizTag pch.quizDefaultTitle

1. Twelve jobs run under `asyncio.Semaphore(4)`. What was the peak number in flight?

   A 12 B 4 C It varies between 1 and 12 D 1
   pch.quizShowAnswer

   B — 4 — Measured at exactly 4. The ceiling is a hard guarantee, not an average — and the same held for limits of 2 and 12.

2. Why must you not use `threading.Semaphore` inside an event loop?

   A It is slower B It blocks the whole thread, including the tasks holding the permits it is waiting for C It is not thread-safe D It cannot be awaited, so it silently does nothing
   pch.quizShowAnswer

   B — It blocks the whole thread, including the tasks holding the permits it is waiting for — That is a deadlock rather than a slowdown. `asyncio.Semaphore` suspends the waiting task and lets the loop run others, so a permit can actually be released.

3. When is an `asyncio.Lock` actually needed?

   A Around any shared variable B Only when the critical section CONTAINS an `await` C Never — asyncio is single-threaded D Only when using `gather`
   pch.quizShowAnswer

   B — Only when the critical section CONTAINS an `await` — There is one thread and control only changes hands at an `await`. Code between awaits cannot be interrupted, so it needs no lock; a read-await-write sequence does.

4. The permit is released when?

   A When the coroutine returns B On leaving the `async with` block, including when the body raises C Only on an explicit `release()` D When the event loop is idle
   pch.quizShowAnswer

   B — On leaving the `async with` block, including when the body raises — That is the point of the context-manager form — an exception inside the body cannot leak a permit and starve everyone else.

🧪 Try It Yourself

Exercise 1 – asyncio.Lock

pch.exerciseTag pch.exerciseTitle

Exercise 2 – asyncio.Semaphore

pch.exerciseTag pch.exerciseTitle

Exercise 3 – asyncio.Event

pch.exerciseTag pch.exerciseTitle