Priority Queue

Overview

A priority queue prevents head-of-line blocking by importance. The robust implementation is separate queues, since in-queue priority flags are weakly supported and hard to observe.

Good policy matters as much as mechanics: decide who may publish at each level and what that level promises.

Broker-native priority support exists in some systems, but with caveats. Priorities are often honored only among messages currently in memory or on a single node, and large backlogs can behave unexpectedly. Separate queues give isolated metrics, separate scaling and clear failure domains, which is why many teams prefer them.

How it works

Workers poll lanes by weight. Each cycle takes more from high lanes but always visits low ones, so progress is guaranteed.

lanes  = {high: 6, normal: 3, low: 1}   # polls per cycle
for lane, n in lanes.items():
    for _ in range(n):
        msg = queue[lane].poll()
        if msg: handle(msg)

Policy deserves as much attention as mechanics. Define what each lane promises, for example high means processed within five seconds at the 99th percentile, and who is allowed to publish there. Rate-limit the top lane so a bug cannot flood it. Also consider reserving a fixed worker slice for high priority so it never waits for capacity to scale up.

Pitfalls

• Strict priority with no minimum share, starving bulk work indefinitely.
• No governance, leading to priority inflation.
• Too many levels that nobody can reason about.
• Autoscaling on total depth, which hides a stuck high lane.

Key takeaways

• Use separate queues for visibility and isolation.
• Guarantee low lanes a minimum share.
• Govern who may use top priority.
• Alert on age of oldest message per lane.