Hot Key Mitigation

Overview

A hot key is one key whose traffic exceeds what a single node can serve. Consistent hashing and partitioning balance the number of keys, not the number of requests, so a single popular key defeats them.

How it works

Read-side replication:

write(k, v):   for i in 0..N-1: cache.set(k + "#" + i, v)
read(k):       i = random(N); return cache.get(k + "#" + i)

Each suffixed key hashes to a different node, so reads divide roughly by N. Updates must touch all copies, so use this for data that changes rarely.

Write-side splitting for counters:

increment(k):  shard = random(16); incr(k + ":" + shard)
total(k):      sum(get(k + ":" + s) for s in 0..15)

A local cache with a one to five second TTL in each application instance is often the cheapest fix: with 200 instances, a node sees at most 200 requests per TTL for that key instead of millions.

Detect hot keys with sampled counters in the proxy or client library, then promote only those keys to replication. Many stores, such as DynamoDB with adaptive capacity, also offer automatic isolation of hot partitions, but only up to per-partition limits.

Pitfalls

• Replicating all keys instead of only the hot ones.
• Forgetting to invalidate every replica on update.
• Splitting a key that requires transactional reads or exact ordering.
• Using a fixed list of hot keys. Popularity moves, so detection must be continuous.
• Rate limits and counters on a single key in front of a global resource.

Key takeaways

• Load skew, not key count, is the problem; more nodes alone will not fix it.
• Layer a short-TTL local cache first, then replicate or split.
• Detect hot keys automatically and apply mitigations selectively.
• Accept brief inconsistency for replicated read copies.