Problem
Incidents are chaotic. No clear process. On-call burns out. Postmortems are blame games. Same incidents recur.
reliability-opsestablished · high operational burden
Incident Response Process
Also known as: incident-management, on-call, postmortem, runbook, blameless
Intent
Structured incident response: detection, response, resolution, postmortem — reducing MTTR and preventing recurrence.
Forces
- •Incidents happen at 3am. Cognitive capacity reduced.
- •Knowledge is distributed, often undocumented
- •MTTR (Mean Time To Resolve) directly impacts revenue
- •Learning from failure requires psychological safety
Solution
✓ When to Use
- •Any production system with on-call
- •Team experiencing burnout or high MTTR
- •Repeated incidents, no learning
✗ When Not to Use
- •Pre-production (lightweight process)
- •Single-person team (simplify)
Pros
- +Reduces MTTR: structure, runbooks, practice
- +Reduces burnout: clear roles, timeboxes, compensation
- +Prevents recurrence: blameless postmortems, action items
- +Builds trust: transparent communication, blameless culture
Cons
- −Process overhead: meetings, docs, reviews
- −On-call burden: compensation, burnout risk
- −Process rigidity: adapt to incident, don't force template
- −Cultural change: blameless takes time, leadership buy-in
Cost Profile
Infrastructure
Low — tools, documentation
Operational
High — on-call, game days, postmortems
Cognitive
Medium — process, culture, communication
Failure Modes
Process theater: follow template, no real learning
Hero mode: IC debugs instead of coordinates
Postmortem theater: template filled, no action items
Alert fatigue: pages for non-SEV-1
Burnout: on-call unsustainable, no compensation
Real-World Examples
Alternatives
- ad-hoc
- ticket-only
- no-process
- chaos-engineering-only
Related Patterns
- runbook
- observability
- alerting
- sli-slo
- chaos-engineering
- game-day
- blameless-postmortem
- on-call
Competency Domains
reliability opsdeploymentdistribution communicationeconomics evolutionsecurity compliance