ReviseAlgo Logo

Concurrency

Thread-Safe Queues

The queue module for thread-safe inter-thread communication — Queue, LifoQueue, PriorityQueue for producer-consumer patterns.

Interview: Producer-consumer pattern is a classic interview problem — queues are the standard solution.

Last Updated: June 12, 2026 • 8 min read

Python's queue module provides thread-safe FIFO, LIFO, and priority queues for communication between threads. They're the standard way to pass data between producer and consumer threads without explicit locking — all operations are internally synchronized.

Queue Types

  • Queue(maxsize=0) — FIFO queue (first in, first out). maxsize=0 means infinite
  • LifoQueue(maxsize) — LIFO/stack (last in, first out)
  • PriorityQueue(maxsize) — items retrieved in priority order (lowest first)
  • SimpleQueue() — unbounded FIFO without task tracking (simpler, faster)

Key Methods

  • put(item, block=True, timeout=None) — add item (blocks if full)
  • get(block=True, timeout=None) — remove and return item (blocks if empty)
  • task_done() — signal that a dequeued task is complete
  • join() — block until all items have been processed (all task_done called)
  • qsize(), empty(), full() — status checks (may be unreliable with threads)

Producer-Consumer Pattern

  • Producers call put() to add work items
  • Consumers call get() in a loop, process items, then call task_done()
  • Main thread calls join() to wait for all work to complete
  • Use a sentinel value (like None) to signal consumers to stop

Interview Insight

Queues are the standard solution for the producer-consumer pattern. The key insight: queues handle synchronization internally — no locks needed. Use task_done() + join() to know when all work is complete. Bounded queues (maxsize > 0) provide natural back-pressure.

Common Pitfall

Don't call qsize() for flow control — it may return stale values in multi-threaded code. Use empty() and full() with timeouts, or better yet, use blocking get()/put() which handle synchronization correctly.

Use Cases

Producer-consumer — decoupling data generation from processing

Worker pools — fixed number of workers processing shared work

Task scheduling — PriorityQueue for priority-based execution

Rate limiting — bounded queues provide natural back-pressure

Pipeline stages — queues connecting processing stages

Common Mistakes

Not calling task_done() — join() will block forever waiting for completion

Forgetting to stop consumers — without sentinels, consumer threads run forever

Using qsize() for flow control — unreliable in multi-threaded context

Unbounded queues with fast producers — memory grows without limit; use maxsize

Not joining consumer threads — main program exits before workers finish