Thread Communication (Queue)

diagram a queue is how threads hand work over without sharing state mermaid

  flowchart LR
  A["producer thread"] --> B["q.put(item)"]
  B --> C{"maxsize reached?"}
  C -->|yes| D["producer BLOCKS -- back-pressure"]
  C -->|no| E["item queued"]
  E --> F["q.get() in a worker"]
  F --> G{"empty?"}
  G -->|yes| H["worker BLOCKS -- no polling, no spinning"]
  G -->|no| I["process it"]
  I --> J["q.task_done()"]
  J --> K["q.join() returns when every put has a matching task_done"]
  L["a sentinel value"] --> M["is how workers are told to stop"]

Two threads touching the same list need a lock and a way to wait. queue.Queue is that lock plus that waiting, already correct: get blocks until an item exists, put blocks when a bounded queue is full, and task_done with join gives you a way to know the work is finished.

Why Queue?

queue.Queue is thread-safe.

Use it to:

• pass tasks from producer threads to worker threads
• avoid manual lock management

Producer-consumer example

queue_example.py

import threading
import queue
import time
 
q = queue.Queue()
 
 
def producer():
    for i in range(5):
        q.put(i)
        print("produced", i)
    q.put(None)  # sentinel
 
 
def consumer():
    while True:
        item = q.get()
        try:
            if item is None:
                break
            print("consuming", item)
            time.sleep(0.2)
        finally:
            q.task_done()
 
 
t1 = threading.Thread(target=producer)
t2 = threading.Thread(target=consumer)
 
t1.start(); t2.start()
 
t1.join()
q.join()  # wait until all tasks are marked done
print("All tasks processed")

Tips

• Use a sentinel (None) to stop consumers.
• Always call task_done() (often in a finally).

sketch A queue is a lock plus a place to wait, already written p5.js Pause Restart

Two threads sharing a list would need a lock and a way to sleep until there is work. queue.Queue is both. The consumer parks when it is empty rather than spinning, and a bounded queue parks the producer when it is full -- which is what stops a fast producer from filling memory.

Check yourself

pch.quizTag pch.quizDefaultTitle

1. A consumer calls `q.get()` on an empty `queue.Queue`. What happens?

   A It returns None B It raises `Empty` C It blocks until an item arrives D It spins in a loop
   pch.quizShowAnswer

   C — It blocks until an item arrives — It parks, using no CPU, until something is put. `get_nowait()` is the version that raises `Empty` instead.

2. What does `maxsize` give you beyond a memory limit?

   A Faster puts B Back-pressure — a full queue blocks the producer C Ordering guarantees D Thread safety
   pch.quizShowAnswer

   B — Back-pressure — a full queue blocks the producer — With `maxsize=0` (unbounded) a producer faster than its consumer will grow the queue until memory runs out. A bound makes the producer wait instead.

3. What is the point of `task_done()` and `join()`?

   A Stopping the worker threads B Knowing when every item that was put has been fully handled C Draining the queue D Resetting the queue
   pch.quizShowAnswer

   B — Knowing when every item that was put has been fully handled — `join()` returns when every `put` has a matching `task_done`. Note it tracks completion of the WORK, not merely that the queue is empty.

4. How do you tell worker threads to stop?

   A Call `q.close()` B Put one sentinel value per worker on the queue C Set them as daemons D Call `thread.stop()`
   pch.quizShowAnswer

   B — Put one sentinel value per worker on the queue — Each worker takes one sentinel and exits, so every worker gets exactly one. A queue has no close, and killing threads is not possible.

🧪 Try It Yourself

Exercise 1 – Put and Get

pch.exerciseTag pch.exerciseTitle

Exercise 2 – Producer-Consumer

pch.exerciseTag pch.exerciseTitle

Exercise 3 – Queue Size

pch.exerciseTag pch.exerciseTitle