Limited Offer

30% OFF Lifetime Access ($139) with code SYSTEM30

TOPIC #59Advanced 9 min read

Multi-Primary (Multi-Master) Replication

💡
Core Architecture Summary

Scale write capacity across geographic regions: Active-Active topologies, write conflicts, Last-Write-Wins (LWW), Vector Clocks, and CRDTs (Conflict-Free Replicated Data Types).

Key Glossary Concepts in this TopicAll Glossary Terms

Multi-Primary Active-Active Replication & Conflict Resolution Strategies 🌍

How Multi-Master systems allow concurrent writes across geographic regions, and how conflicts are resolved via Last-Write-Wins (LWW), Vector Clocks, and CRDTs.

Multi-Primary Active-Active Replication & Conflict Resolution Strategies 🌍
100%
Rendering visual architecture flowchart...

01.1. What is Multi-Primary (Active-Active) Replication?

In standard Primary-Replica systems, all writes must travel to a single geographic Primary node, forcing global users in Australia or Europe to endure high-latency cross-continental network hops (~150ms) for every write mutation.

Multi-Primary (Multi-Master / Active-Active) Replication deploys two or more independent Primary nodes located in different data centers or geographic regions:

  • Every Primary node can process both Read and Write transactions locally in <5ms.
  • Asynchronously or synchronously propagates writes bi-directionally across the global cluster.
  • The Immense Challenge: What happens when two users concurrently update the exact same database row on different Primary nodes at the exact same millisecond?

02.2. The Write-Write Conflict Dilemma

Consider an account with balance $100:

  • User A in New York withdraws 50 on US-Primary (sets balance =50).
  • User B in London concurrently withdraws 60 on EU-Primary (sets balance =40).
  • Both Primary nodes accept the writes locally and acknowledge success.
  • When the asynchronous cross-region replication streams meet across the Atlantic, the database encounters a Write-Write Conflict. Overwriting one balance destroys money; keeping both creates an inconsistent split-brain.

03.3. The 4 Conflict Resolution Strategies

Distributed systems resolve concurrent multi-primary conflicts using four distinct strategies:

1. Last-Write-Wins (LWW / Timestamp-Based)

The database attaches a physical wall-clock timestamp to every write. When a conflict occurs, the write with the highest timestamp overwrites the older write.

  • The Danger: NTP Clock Drift. If Server A's clock is 100ms fast, its writes will silently overwrite and destroy legitimate updates from Server B, causing silent data loss.

2. Conflict-Free Replicated Data Types (CRDTs)

CRDTs are specialized mathematical data structures that can be replicated concurrently across multiple independent nodes without central coordination, guaranteed to converge to the exact same state regardless of the order or interleaving of received operations:

  • PN-Counters (Positive-Negative Counters): Used for shopping cart quantities, likes, and voting counts.
  • LWW-Element-Set / Observed-Remove Set (OR-Set): Used for collaborative real-time documents (Figma, Google Docs, Apple Notes).

3. Vector Clocks & Version Vectors

Attaches a tuple of counters [NodeA: 2, NodeB: 5] to each record. Allows the database to detect whether two updates are causally related (A → B) or concurrent (conflict). Concurrent branches are stored simultaneously (like a Git merge conflict), requiring application code to merge the values.

4. Conflict Avoidance (Geographic Data Partitioning)

Avoid conflicts altogether by ensuring that a specific customer's record is strictly owned by exactly one region:

  • US users are pinned to US-Primary; EU users are pinned to EU-Primary. Cross-region writes are prohibited or coordinated via global locks.

⚖️Architectural Trade-offs & Production Realities

Architectural Advantages

  • Delivers sub-5ms local write latency for global users worldwide.
  • Extreme high availability: if an entire AWS cloud region suffers an outage, other regional primaries continue accepting writes without failover delays.
  • CRDT data structures eliminate manual conflict resolution for collaborative tools.

Trade-offs & Constraints

  • High complexity: handling write conflicts, clock drift, and schema migrations across active-active masters is notoriously difficult.
  • Last-Write-Wins (LWW) risks silent data loss during concurrent mutations.
Production Implementation in Big Tech
Figma & Apple Notes• Collaborative Multi-Device State Synchronization via CRDTs

Figma and Apple Notes use Conflict-Free Replicated Data Types (CRDTs) to allow multiple users to edit documents concurrently across web browsers and mobile devices offline, merging conflicting edits into mathematically identical state without losing text or canvas elements.

🎯 Staff+ Engineering Takeaways

  • Multi-Primary allows multiple nodes to process writes concurrently in different regions.
  • Concurrent writes on different primaries create Write-Write Conflicts.
  • LWW (Last-Write-Wins) uses timestamps but is vulnerable to NTP clock skew.
  • CRDTs provide mathematical data convergence for distributed collaborative structures.
  • Conflict Avoidance (pinning user data to specific regions) is the safest architectural pattern.

Topic Knowledge Assessment 🧠

Step through 2 scenario questions to test your staff-level grasp.

Question 1 of 20 answered
#1

What is the primary vulnerability of relying on Last-Write-Wins (LWW) with physical timestamps for resolving multi-master database conflicts?

Rate This Architecture Chapter4.9 / 5.0 (38 ratings)

How clear and staff-actionable was this system breakdown?