A message queue lets one part of your system hand off work to another without waiting for it to finish. This decouples services, smooths traffic spikes, and makes systems more resilient — a fundamental building block in large designs.
The producer-consumer model
A producer puts a message on the queue; a consumer reads and processes it later. The producer doesn't wait for the work to complete — it just needs the message to be accepted. If consumers are slow, messages simply wait in the queue instead of dropping requests.
Why use a queue
- Decoupling — services don't need to know about each other or be online at the same time.
- Load levelling — absorb spikes; consumers drain the backlog at their own pace.
- Resilience — if a consumer crashes, messages stay in the queue and get retried.
- Async work — offload slow tasks (emails, image processing, analytics) so the user request returns fast.
Queues vs pub/sub
A queue delivers each message to one consumer (work distribution). Publish/subscribe broadcasts each message to every subscriber (fan-out). Kafka is a log-based system supporting both patterns at huge scale; RabbitMQ is a classic message broker.
Delivery guarantees
Most systems offer at-least-once delivery, which means duplicates are possible — so consumers should be idempotent (processing the same message twice has no extra effect). Exactly-once is expensive and rare; at-most-once risks losing messages.