Stackbook Logo
deploymentestablished · medium operational burden

Progressive Delivery

Also known as: gradual-rollout, staged-rollout

Intent

Release changes to progressively larger audiences while measuring impact, with automatic halting or rollback when metrics degrade.

Problem

Releasing to everyone at once turns every bug into a full outage, and deploy and release are the same irreversible event. Teams either ship slowly out of fear or ship fast and absorb incidents.

Forces

  • •Faster delivery increases exposure to defects
  • •Real traffic reveals problems tests miss
  • •Rollback needs to be quick and ideally automatic
  • •Different cohorts carry different risk

Solution

Overview

Progressive delivery extends continuous delivery with controlled exposure. A deployment puts new code on servers; a release exposes it to users. Splitting the two lets teams ship constantly and expose cautiously.

How it works

A rollout is a sequence of steps, each with a traffic share and a health gate.

steps:
  - weight: 5     pause: 10m   gate: error_rate < 1%
  - weight: 25    pause: 15m   gate: p99 < 400ms
  - weight: 50    pause: 15m
  - weight: 100
on gate failure: abort -> weight 0

The controller shifts traffic using a load balancer, service mesh or flag service. It queries a metrics backend during each pause and compares canary against stable. Failure triggers rollback without paging a human.

Pitfalls

  • Gates that only watch HTTP errors miss correctness bugs, such as wrong prices. Add business metrics.
  • Too little traffic in the first stage to see anything.
  • Both versions writing to the same schema without compatibility. Pair with expand-contract migrations.
  • Sticky sessions or caches that mask which version served a request.
  • Never cleaning up flags and old rollout configuration.
  • Treating the pipeline as safe when the rollback path is untested.

Key takeaways

  • Deploy is not release; control exposure separately.
  • Define explicit health gates before starting.
  • Automate abort, not just promotion.
  • Make versions mutually compatible for the overlap window.
  • Invest in metrics quality; the pattern is only as good as its signals.

✓ When to Use

  • •High-traffic services where defects are costly
  • •Teams deploying many times per day
  • •Changes with uncertain performance or behavioural impact

✗ When Not to Use

  • •Services with too little traffic for statistically meaningful signals
  • •Changes that are all-or-nothing, such as incompatible schema flips
  • •No usable metrics or alerting to judge health

Pros

  • +Limits blast radius of defects
  • +Decouples deploy from release
  • +Enables data-driven promotion

Cons

  • −Needs good observability and traffic control
  • −Multiple versions run concurrently
  • −Longer time to full rollout

Cost Profile

Infrastructure

Medium - extra capacity and routing layer

Operational

Medium - tuning analysis rules and flags

Cognitive

Medium - concurrent version compatibility

Failure Modes

  • Metrics too noisy so bad releases pass

  • Incompatible versions corrupt shared state while both run

  • Flag and rollout leftovers pile up as technical debt

Real-World Examples

Alternatives

  • blue-green-deployment

Related Patterns

  • canary-deployment
  • feature-flag
  • canary-analysis

Competency Domains

reliability opseconomics evolution