Stackbook Logo
reliability-opsestablished · medium operational burden

Error Budget Policy

Also known as: error-budgets, reliability-budget

Intent

Convert an SLO into a spendable budget of allowed failure and tie it to explicit decisions about feature work and reliability work.

Problem

Product teams want to ship fast and operations teams want stability, and each side argues from opinion. Without a shared measure, reliability work is either neglected or over-invested.

Forces

  • •Velocity and reliability compete for the same engineering time
  • •100 percent reliability is neither achievable nor wanted
  • •Decisions need objective evidence
  • •Policy must be agreed before an incident, not during one

Solution

Overview

An error budget is the amount of unreliability an SLO permits. If the SLO is 99.9 percent success over 30 days, the budget is 0.1 percent of requests, or about 43 minutes of full downtime. The budget turns reliability into a currency teams can spend on releases, experiments, and migrations.

How it works

Budget consumption is computed from the SLI:

budget_total     = (1 - SLO) * events_in_window
budget_consumed  = bad_events_in_window
budget_remaining = 1 - budget_consumed / budget_total

Policy example
  remaining above 50 percent : ship normally
  remaining 0 to 50 percent  : require canary and rollback plan
  remaining at 0             : freeze features, fix reliability

The policy document is agreed in advance by product, engineering, and operations, and names an escalation path when parties disagree. Exceptions, such as security patches, are listed in the policy. A burn-rate alert warns before the budget is gone.

Pitfalls

  • Writing a policy that nobody enforces. The budget becomes a decorative graph.
  • Choosing an SLO from aspiration instead of user need. A 99.99 percent target for a tolerant internal tool wastes effort.
  • Counting outages caused by third parties against the team without discussing whether the SLO accounts for them.
  • Using a window so long that old incidents mask current problems, or so short that one blip exhausts it.

Key takeaways

  • The budget is one minus the SLO, measured over a fixed window.
  • A pre-agreed policy removes argument during crunch time.
  • Spend the budget deliberately; unused budget suggests too much caution.
  • Pair it with burn-rate alerts for early warning.

✓ When to Use

  • •Services with a defined SLO and an active release cadence
  • •Resolving velocity-versus-stability disputes
  • •Prioritizing reliability backlog items

✗ When Not to Use

  • •Services with no reliable SLI measurement
  • •Safety-critical systems where any failure is unacceptable
  • •Teams without management support to honor the freeze

Pros

  • +Objective basis for release decisions
  • +Legitimizes taking calculated risks
  • +Aligns product and engineering incentives

Cons

  • −Requires trustworthy SLIs
  • −Freezes can be politically hard to enforce
  • −Poorly chosen SLOs produce meaningless budgets

Cost Profile

Infrastructure

Low - computed from existing SLIs

Operational

Medium - policy reviews and enforcement

Cognitive

Medium - needs organizational buy-in

Failure Modes

  • Policy not enforced, so the budget has no effect

  • SLO set too tight, leaving a permanently exhausted budget

  • Budget spent by one dependency's outage, causing unfair blame

Real-World Examples

Alternatives

    Related Patterns

    • sli-slo
    • incident-response

    Competency Domains

    reliability opseconomics evolution