Problem
Product teams want to ship fast and operations teams want stability, and each side argues from opinion. Without a shared measure, reliability work is either neglected or over-invested.
Also known as: error-budgets, reliability-budget
Convert an SLO into a spendable budget of allowed failure and tie it to explicit decisions about feature work and reliability work.
Product teams want to ship fast and operations teams want stability, and each side argues from opinion. Without a shared measure, reliability work is either neglected or over-invested.
An error budget is the amount of unreliability an SLO permits. If the SLO is 99.9 percent success over 30 days, the budget is 0.1 percent of requests, or about 43 minutes of full downtime. The budget turns reliability into a currency teams can spend on releases, experiments, and migrations.
Budget consumption is computed from the SLI:
budget_total = (1 - SLO) * events_in_window
budget_consumed = bad_events_in_window
budget_remaining = 1 - budget_consumed / budget_total
Policy example
remaining above 50 percent : ship normally
remaining 0 to 50 percent : require canary and rollback plan
remaining at 0 : freeze features, fix reliability
The policy document is agreed in advance by product, engineering, and operations, and names an escalation path when parties disagree. Exceptions, such as security patches, are listed in the policy. A burn-rate alert warns before the budget is gone.
Low - computed from existing SLIs
Medium - policy reviews and enforcement
Medium - needs organizational buy-in
Policy not enforced, so the budget has no effect
SLO set too tight, leaving a permanently exhausted budget
Budget spent by one dependency's outage, causing unfair blame