On-Call Rotation Design

Overview

On-call design decides who responds, when, and with what support. A sound design treats human attention as a limited resource: it caps load, spreads knowledge, and makes escalation unambiguous so that nobody is stuck alone during a serious incident.

How it works

A typical layout has a primary responder, a secondary who is paged if the primary does not acknowledge, and an incident manager or manager-on-call for escalation.

Minute 0   alert fires, page primary
Minute 5   no ack, page secondary
Minute 15  no ack, page engineering manager
Sev1       primary declares incident, pulls incident commander

Weekly handoff note
  - open incidents and workarounds
  - flaky alerts seen
  - risky deploys scheduled

Metrics guide staffing: pages per shift, share of pages outside working hours, percent actionable, and acknowledgement time. Many teams aim for roughly one or two incidents per twelve-hour shift at most; sustained above that signals a reliability problem, not a staffing one. New members shadow a shift and then reverse shadow before taking the pager. Each service has a runbook linked from the alert.

Pitfalls

• Rotations of three or four people, which mean on-call every few weeks and little recovery time.
• Paging for non-actionable alerts. Every page should need a human and have a runbook.
• No protected recovery after a bad night. Allow time off the next day.
• Treating on-call as unpaid extra work, which drives out experienced engineers.

Key takeaways

• Use a rotation large enough to recover between shifts.
• Define escalation with timeouts.
• Measure load and use it to justify reliability work.
• Support responders with runbooks, shadowing, and compensation.