Scatter-Gather

Overview

Scatter-gather is a messaging and orchestration pattern in which one request fans out to many recipients and one aggregator fans the replies back in. It is the backbone of distributed search, price comparison and multi-shard queries.

How it works

The coordinator assigns a correlation id, sends the request to every recipient in parallel and waits under a deadline.

request(id=77) --> scatter --> shard-1 --\
                          \--> shard-2 ---+--> gather(id=77) --> merge/rank --> response
                           \-> shard-3 --/
completion: all replies OR quorum OR deadline 200 ms

Completion policy is the key design choice. Search often waits for the deadline and returns what arrived, flagging partial results. Price comparison may wait for all vendors up to a timeout. Latency hedging sends the same request to replicas and takes the first answer, cancelling the rest. The gatherer discards late replies by correlation id.

Pitfalls

• Waiting for every responder. Always set a deadline and decide what partial means to the caller.
• Load amplification. A fan-out of 50 turns one request into 50; apply concurrency limits and circuit breakers per recipient.
• Tail latency. The more responders, the more likely one is slow; use hedging or drop stragglers.
• Orphaned work. Cancel outstanding calls once the answer is complete.
• Inconsistent ranking when sources use different scoring scales; normalise before merging.

Key takeaways

• Parallel fan-out, correlated fan-in, bounded by a deadline.
• Choose completion policy deliberately: all, quorum or first.
• Protect recipients from amplified load.
• Make partial results explicit to callers.