Synchronization in Python

What is synchronization?

Synchronization is how you coordinate multiple threads/processes so they:

• don’t corrupt shared data
• don’t run in the wrong order
• can safely communicate

Why it matters

Without synchronization you can get:

• race conditions
• deadlocks
• inconsistent reads

Common synchronization tools

For threads (threading)

• Lock, RLock
• Semaphore, BoundedSemaphore
• Event
• Condition
• Barrier

For processes (multiprocessing)

• multiprocessing.Lock, RLock
• multiprocessing.Semaphore
• multiprocessing.Event
• multiprocessing.Condition

Design tip

Prefer designs that reduce shared mutable state:

• message passing via queue.Queue (threads)
• multiprocessing.Queue (processes)

When you must share state, use synchronization.

Why counter += 1 is three operations

A single statement compiles to a read, an add, and a write. A thread can be suspended between any two of them, and when it resumes it writes back a value computed from stale data:

  sequenceDiagram
  participant A as thread A
  participant M as counter (memory)
  participant B as thread B
  Note over M: counter = 5
  A->>M: read -> 5
  B->>M: read -> 5
  A->>A: add 1 -> 6
  B->>B: add 1 -> 6
  A->>M: write 6
  B->>M: write 6
  Note over M: counter = 6, not 7
one increment vanished

Both threads did their work. One increment is gone, nothing raised, and the total is simply wrong.

The uncomfortable part: you probably cannot reproduce it

Four threads, 200,000 increments each, plain counter += 1 on CPython 3.14.4:

The first two rows are the important ones. The race is genuinely present in that code — nothing about it is thread-safe — and it still lost nothing across 800,000 opportunities, because the gap between the read and the write is only a few bytecodes wide and the interpreter rarely switches inside it.

Forcing a yield into that gap loses 75% immediately, and gives the same figure on CPython 3.11.9, so this is not a quirk of one version.

import threading
 
counter = 0
lock = threading.Lock()
 
def worker(n):
    global counter
    for _ in range(n):
        with lock:            # the whole read-modify-write is now indivisible
            counter += 1

See it move

Step the two threads by hand and try to lose an increment. With the lock on, the second thread cannot enter until the first has written back.

The other primitives, measured

Lock is not reentrant. RLock is.

lk = threading.Lock()
lk.acquire()
lk.acquire(timeout=0.3)      # False — the SAME thread cannot acquire it twice
                             # without a timeout this deadlocks against itself
 
rl = threading.RLock()
rl.acquire()
rl.acquire(timeout=0.3)      # True — counts recursion, needs matching releases

A method that takes a lock and calls another method that takes the same lock will hang on a plain Lock. That is the case RLock exists for.

Semaphore(n) caps how many run at once. 12 jobs of 0.20 s through Semaphore(3): measured peak concurrency exactly 3.

Event releases every waiter at once, not one:

ev = threading.Event()
# four threads call ev.wait()
# before ev.set(): 0 released
ev.set()
# after  ev.set(): 4 released   <- all of them

Use an Event for "the configuration is loaded, everyone may proceed". Use a Semaphore when you want to admit a fixed number at a time.

Deadlock needs only two locks and two orders

def ab():
    with A:
        with B:  ...          # A then B
 
def ba():
    with B:
        with A:  ...          # B then A   <- opposite order

Measured with each thread guaranteed to be holding its first lock: the second acquire timed out, which is what a deadlock looks like when you give it a deadline. Without timeout=, both threads wait forever, consuming no CPU — a process that is perfectly healthy by every metric and doing nothing at all.

Check yourself

🧪 Try It Yourself

Exercise 1 – Basic Lock Usage

Press Run to see what your code prints.

Exercise 2 – Lock as Context Manager

Press Run to see what your code prints.

Exercise 3 – Detect Locked State

Press Run to see what your code prints.