Retry Pattern Best Practices

Retrying is the simplest resilience technique and the easiest to misuse. Done well, it hides transient noise. Done badly, it multiplies traffic exactly when a dependency can least afford it.

How it works

After a failure classified as retryable, the client waits for a delay computed as the base interval times two to the power of the attempt, capped at a maximum, then randomized with jitter. It stops when the attempt count or total deadline is reached. A retry budget, such as ten percent of recent requests, prevents retries from dominating traffic. If the response includes a Retry-After header, the client honors it. Only the outermost layer that has context about the user request performs the retry, avoiding multiplication across tiers.

Implementation steps

1. Classify errors as retryable or permanent
2. Confirm the operation is idempotent or protected by a key
3. Choose a small attempt limit and an overall deadline
4. Apply exponential backoff with full or decorrelated jitter
5. Add a retry budget and honor server back-pressure signals
6. Retry at a single layer and emit metrics for every retry

Common pitfalls

• Retrying without jitter, creating synchronized spikes
• Retries at every tier, causing multiplicative amplification
• Retrying timeouts of non-idempotent writes
• Ignoring the caller's own deadline, so work continues after the user has left

Metrics to watch

• Retry rate as a fraction of total requests
• Success rate by attempt number
• Added latency from retries
• Retry budget exhaustion events
• Server load during dependency incidents

Example configuration

// Exponential backoff with full jitter
async function withRetry(fn, { max = 4, base = 100, cap = 5000 } = {}) {
  for (let attempt = 0; ; attempt++) {
    try {
      return await fn();
    } catch (err) {
      if (!err.retryable || attempt >= max) throw err;
      const delay = Math.random() * Math.min(cap, base * 2 ** attempt);
      await new Promise((r) => setTimeout(r, delay));
    }
  }
}

Trade-offs and adoption

Teams adopt this approach when calls fail transiently due to network or throttling; operations are idempotent or protected by idempotency keys. It is a poor fit when failures are permanent, such as validation or authorization errors, or when operations are non-idempotent and cannot be keyed. The central tensions are success rate vs added load; persistence vs latency; client autonomy vs system-wide retry budgets. The main benefits are that it delivers: masks brief faults from users; simple to implement with libraries; improves success rates for flaky networks; backoff and jitter reduce synchronized load. The main costs to budget for are: amplifies load during widespread failures; adds latency to failing requests; risk of duplicate effects without idempotency; layered retries multiply exponentially. Governance typically looks like this: shared client libraries encode approved retry defaults. Expect adoption cost to be low: library configuration and policy, and a best fit for any team making remote calls. Critics point out that retries are a leading cause of metastable failures and retry storms, and that defaults copied between services often ignore each dependency's behavior. Treat these concerns as design inputs, and revisit the decision as traffic, team size, and failure history change.

Key takeaways

• Retry only transient errors on idempotent operations
• Always use backoff with jitter and an overall deadline
• Limit retries with a budget and a single retry layer
• Monitor retry volume as a warning sign