Split-Brain & Fencing Mechanisms

In distributed architectures with primary-follower or active-passive topologies, high availability depends on automatically promoting a standby node to primary whenever the active primary crashes.

However, consider what happens when a network partition severs communication between two datacenters (e.g., Datacenter 1 and Datacenter 2) while all individual machines remain healthy:

• Datacenter 1 (Old Master): Believes Datacenter 2 is down and continues accepting client writes.
• Datacenter 2 (Standby): Heartbeats from Datacenter 1 time out; Datacenter 2 assumes the primary has crashed and promotes its local node to Primary.
• The Split-Brain State: Both nodes now believe they are the authoritative Primary. Both accept conflicting INSERT, UPDATE, and DELETE transactions from clients.

When the network partition heals, the two database datasets have diverged irreparably with conflicting primary keys and modified balances, resulting in irrecoverable data corruption.