Stackbook Logo
resilienceestablished · medium operational burden

Shuffle Sharding

Also known as: virtual-shards, blast-radius-partitioning

Intent

Assign each tenant a small random subset of workers so a noisy or poison tenant affects only a tiny fraction of others.

Problem

In a shared fleet one misbehaving client, such as a request of death or a traffic flood, can take down every worker and hurt all tenants. Plain sharding limits blast radius to one shard, but each shard still affects many customers.

Forces

  • •Shared capacity is cost efficient
  • •Tenants vary in behavior
  • •Isolation needs more partitions than workers
  • •Clients must be able to retry on another worker

Solution

Overview

Shuffle sharding applies combinatorics to isolation. Instead of dividing workers into a few fixed shards, each tenant gets its own random combination of workers, so overlap between any two tenants is rare. Amazon Route 53 uses it to contain denial-of-service attacks.

How it works

Choose a subset size k out of n workers, derived from a hash of the tenant id.

workers: w1 w2 w3 w4 w5 w6 w7 w8
tenant A -> w2, w7
tenant B -> w1, w5
tenant C -> w2, w5

If tenant A sends a poison request that crashes workers, only w2 and w7 fail. Tenant B is untouched. Tenant C loses w2 but keeps w5, and a retrying client simply moves to the remaining worker. With 100 workers and subsets of five, there are about 75 million combinations, so a fully shared fate is almost impossible.

Pitfalls

  • The client or router must retry on another member of the subset, or partial failure still shows up as errors.
  • A subset must carry the tenant load alone in the worst case, so size k accordingly.
  • Hash-based assignment can place two heavy tenants on the same workers. Allow overrides.
  • Fleet changes reshuffle assignments, so use consistent hashing or a stored mapping.
  • Per-tenant metrics are necessary to see which tenant is hurting its subset.

Key takeaways

  • Randomized subsets turn blast radius from a fraction of the fleet into a fraction of a fraction.
  • Pair with client retries across the subset.
  • Size subsets for one tenant's peak load.
  • Keep assignment stable as the fleet changes.

✓ When to Use

  • •Multi-tenant services with shared worker fleets
  • •DNS, queue, and API platforms exposed to untrusted traffic
  • •Protecting against request-of-death failures

✗ When Not to Use

  • •Single-tenant systems
  • •Very small fleets with too few combinations
  • •Workloads needing global load balancing for efficiency

Pros

  • +Exponentially smaller blast radius than plain sharding
  • +Efficient shared capacity
  • +Works against both accidents and attacks

Cons

  • −More complex routing and client logic
  • −Uneven load when subsets overlap
  • −Needs capacity per subset to be sufficient

Cost Profile

Infrastructure

Low, uses existing capacity

Operational

Medium, routing and observability per tenant

Cognitive

High, combinatorial reasoning

Failure Modes

  • Subset too small so one tenant failure removes all its capacity

  • Clients do not retry across subset members

  • Hash collisions overlap a heavy tenant with a critical one

Real-World Examples

Alternatives

    Related Patterns

    • bulkhead

    Competency Domains

    reliability opsdistribution communication