The Big Little Guide to Message Queues
sudhir.io/the-big-little-guide-to-message-queues
What Jamie wrote
Message queues have been part of every platform I've been around on the Internet. They are a core building block of any distributed systems, and they are more complicated than it may seem, as well as incredibly powerful.
In this guide, we'll talk about:
- What message queues are and their history.
- Why they're useful and what mental models to use when reasoning about them.
- Delivery guarantees that the queuing systems make (at-least-once, at-most-once, and exactly-once semantics).
- Ordering and FIFO guarantees and how they effect sequencing, parallelism and performance.
- Patterns for fan-out and fan-in: delivering one message to many systems or messages from many systems into one.
- Notes on the pros and cons of many popular systems available today.
This article is a great summary and covers the critical points of message queues. More than a couple of times I've had developers approach me, frustrated after dealing with some complicated queue problem, and suggest that we should "build our own" message queue. In the future, I'll just refer them to this article and highlight all of the complexities that make queues so difficult to build and to run.