The most important thing to understand about queues – Dan Slimmon
blog.danslimmon.com/2016/08/26/the-most-important-thing-to-understand-about-queues
What Jamie wrote
Every major system I've worked with for the last two decades plus has used queues in some way, and anyone that has worked around queues in production long enough has learned the hard way that queues behave very differently as they get larger. Typically queue performance decreases exponentially as the queue gets more messages on it.
To reiterate:
As you approach maximum throughput, average queue size – and therefore average wait time – approaches infinity.
What’s the solution, then? We only have so many variables to work with in a queueing system. Any solution for the exploding-queue-size problem is going to involve some combination of these three approaches:
- Increase capacity
- Decrease demand
- Set an upper bound for the queue size
I like number 3. It forces you to acknowledge the fact that queue size is always finite, and to think through the failure modes that a full queue will engender.
I agree with the author here, not limiting a queue at some point is the equivalent of ignoring error messages. Extra credit can be achieved here by introducing some chaos engineering. How about having your queue server reject 0.1% requests all the time, and then enforce the maximum.
The complicated aspect here is that the queue server cannot determine the maximum, because it dramatically will change with the size and type of messages on the queue.