Stackbook Logo
scalingestablished · medium operational burden

Autoscaling

Also known as: elastic-scaling, auto-scaling-groups

Intent

Adjust compute capacity automatically in response to load so cost and performance track demand.

Problem

Fixed capacity is either over-provisioned and wasteful or under-provisioned and slow during peaks. Manual scaling reacts too slowly to traffic spikes and is forgotten after they pass.

Forces

  • •Demand varies by time of day and events
  • •New instances take time to start and warm up
  • •Scaling signals are noisy and lagging
  • •Downstream systems may not scale with the tier

Solution

Overview

Autoscaling is a feedback loop: measure load, compare to a target, change instance count. It works well when instances are stateless, start quickly and the scaling metric reflects real saturation.

How it works

desired = ceil(current * (observedMetric / targetMetric))

This is the formula used by the Kubernetes Horizontal Pod Autoscaler. If 10 pods run at 90 percent CPU against a target of 60 percent, the desired count is 15. Policies then clamp the result between min and max and apply stabilization windows, commonly short for scale-out and several minutes for scale-in.

Choose metrics that lead rather than lag. CPU works for compute-bound services, but requests in flight per instance or queue length per worker respond earlier for I/O-bound ones. Keep headroom: startup time of 60 to 120 seconds means you must scale before saturation, not at it. Scheduled actions pre-scale for known events.

Pitfalls

  • Scaling on a metric that does not fall when you add instances, such as database latency, causes runaway growth.
  • Aggressive scale-in that kills instances holding in-flight work. Use connection draining and termination hooks.
  • No maximum, or a maximum above downstream capacity. Pair with rate limits and load shedding.
  • Cold instances serving traffic before caches and JIT are warm. Use readiness checks and slow start.
  • Ignoring account quotas and spot capacity limits during regional events.

Key takeaways

  • Scale on a leading metric tied to saturation and keep spare headroom.
  • Set min, max, and different speeds for scaling out and in.
  • Verify downstream dependencies can survive the maximum fleet size.
  • Test the policy with load tests, not assumptions.

✓ When to Use

  • •Stateless services with variable traffic
  • •Queue consumers scaled on backlog per worker
  • •Workloads with predictable daily or weekly peaks

✗ When Not to Use

  • •Stateful systems that need slow, careful rebalancing
  • •Constant-load workloads where reserved capacity is cheaper
  • •When the bottleneck is a shared database that will not scale

Pros

  • +Matches cost to demand
  • +Handles unplanned spikes without paging
  • +Improves availability by replacing unhealthy instances

Cons

  • −Scaling lags demand by startup time
  • −Poor metrics cause oscillation or runaway cost
  • −Pushes load onto downstream dependencies

Cost Profile

Infrastructure

Variable; pay for peak only when needed

Operational

Medium: tune policies and review limits

Cognitive

Medium: control-loop behavior

Failure Modes

  • Flapping from short cooldowns causes constant churn

  • Scale-out overloads the database or hits quota limits

  • Max limit reached during an attack, producing a huge bill or an outage

Real-World Examples

Alternatives

    Related Patterns

    • capacity-planning
    • kubernetes-patterns
    • load-balancing

    Competency Domains

    economics evolutionreliability ops