Cache Stampede Protection

Overview

A cache stampede, or dogpile, happens when many requests find the same key missing and all rebuild it. The cache existed to protect the database, and at the moment of expiry it does the opposite.

How it works

get(key):
  entry = cache.get(key)
  if entry and now < entry.softExpiry: return entry.value
  if cache.setnx("lock:" + key, 1, ttl=10):
      value = loader(key)                      # only one caller
      cache.set(key, wrap(value, soft=now+60), ttl=120)
      cache.delete("lock:" + key)
      return value
  if entry: return entry.value                 # serve stale
  wait briefly, then retry get(key)

The entry carries a soft expiry and a longer hard TTL. Between them, readers get the stale value while exactly one caller refreshes. Without stale data to serve, losing callers wait or poll the cache for a short time.

Probabilistic early expiration needs no lock: each reader recomputes when now minus (delta times beta times the log of a random number) passes expiry, where delta is the last compute time. Slow computations start refreshing earlier.

Pitfalls

• Lock without a TTL, so a crashed holder blocks the key forever.
• Waiters polling in a tight loop, which recreates the load on the cache.
• Identical TTLs for bulk-loaded keys. Add 10 to 20 percent jitter.
• Ignoring failed refreshes. Alert on entries older than the hard TTL target.
• Locks that are only per-process, so many instances still stampede together.

Key takeaways

• Jitter TTLs, collapse concurrent loads, and serve stale while refreshing.
• Always put an expiry on locks.
• Pre-warm critical keys after deploys.
• Measure origin query rate per key, not only hit ratio.