Jitter

Overview

Jitter breaks synchronization. Exponential backoff alone spaces retries out for one client, but a thousand clients that failed together still retry together at each step. Randomness spreads them across the interval.

How it works

The AWS Architecture Blog compared several variants. Full jitter performs well for most retry cases.

cap   = min(maxDelay, base * 2^attempt)
sleep = random(0, cap)            # full jitter

# decorrelated jitter
sleep = min(maxDelay, random(base, prev * 3))

The same idea applies outside retries. Add plus or minus ten percent to cache TTLs so entries do not expire together, offset cron jobs by a random number of seconds, and stagger client reconnects after a server restart.

Pitfalls

• A fixed seed, or seeding from the second, makes instances pick identical delays. Use a proper per-process source of randomness.
• Small jitter, such as plus or minus one percent, does not break up large synchronized groups.
• Jitter can push a call beyond its timeout budget, so cap total delay.
• Tests become flaky unless the random source is injectable.
• Jitter does not limit total retry volume. Combine it with a retry budget.

Key takeaways

• Randomize retries, TTLs, and periodic jobs.
• Prefer full or decorrelated jitter over fixed backoff.
• Use independent randomness per instance.
• Keep the total delay inside the request budget.
• Verify the effect in load tests by plotting request arrivals after a simulated outage, where jittered clients should form a smooth ramp rather than spikes.