Active-Passive Failover

Overview

In active-passive failover one node does the work while another waits. It trades idle capacity for a simple consistency story, since only one node ever accepts writes.

How it works

Replication keeps the standby current, and a detector watches the primary.

clients -> VIP -> [primary] --replicate--> [standby]
primary stops heartbeats
  1. detector confirms (quorum)
  2. fence old primary (STONITH or token)
  3. promote standby, move VIP
  4. old primary rejoins as standby

Synchronous replication avoids data loss but adds latency and can block writes if the standby is down. Asynchronous replication keeps writes fast but may lose the last few transactions. Tools such as Patroni, Pacemaker, and cloud managed databases implement this loop.

Pitfalls

• Without fencing, a network partition produces two primaries and divergent data.
• Automatic failback often causes a second outage, so prefer deliberate failback.
• A standby that is never exercised may fail on first use. Run scheduled failover drills.
• Clients with cached DNS or long-lived connections keep talking to the old primary.
• Capacity on the standby must match the primary, otherwise promotion causes overload.

Key takeaways

• Decide the acceptable data loss and recovery time first.
• Always fence the old primary.
• Use a quorum-based detector to avoid false promotion.
• Test failover regularly and avoid automatic failback.
• Record the measured recovery time and data loss from each drill and compare them with the stated objectives, since real numbers usually differ from design estimates.