Limited Offer

30% OFF Lifetime Access ($139) with code SYSTEM30

TOPIC #79Intermediate 7 min read

Read-Your-Writes & Session Consistency

💡
Core Architecture Summary

Prevent user confusion in eventually consistent systems: Sticky routing, version timestamps, and client-side session tokens.

Key Glossary Concepts in this TopicAll Glossary Terms

Read-Your-Writes Anomaly vs Resolution 🔄

User submits a profile update but reads from a lagging replica.

Read-Your-Writes Anomaly vs Resolution 🔄
100%
Rendering visual architecture flowchart...

01.1. The Read-Your-Writes Anomaly (The User-Experience Disaster)

In modern web architectures utilizing asynchronous primary-replica replication or eventually consistent distributed databases, replication lag is normal (50ms to 500ms).

However, this creates severe user experience bugs:

  1. A user updates their profile picture or comments on a forum post.
  2. The write is committed to the Primary Database, returning an HTTP 200 OK.
  3. The client application immediately triggers a page re-render, dispatching a GET request.
  4. The Load Balancer routes the read request to an asynchronous Read Replica that is 200ms behind the primary.
  5. The replica returns the old profile picture or shows that the comment does not exist.
  6. The user assumes their action failed, clicks the submit button multiple times, and files a bug report.

Read-Your-Writes Consistency (Session Consistency) guarantees that a given user session will always immediately observe the effects of their own updates, even if other users on the platform observe them with slight replication delay.

02.2. Implementation Patterns for Read-Your-Writes Consistency

System designers implement session consistency using three primary architectural patterns:

1. Write Log Sequence Number (LSN) / Version Tracking (Gold Standard)

  • When the primary database commits a write, it returns the Log Sequence Number (LSN) or transaction version timestamp (T_{commit}).
  • The API Gateway passes this metadata back to the client browser in an HTTP response header or session cookie: Set-Cookie: min_lsn=894321; HttpOnly.
  • On subsequent read queries, the client forwards the cookie. The load balancer / proxy inspects available read replicas:
    • If (replica.current_lsn \ge min_lsn), the read routes to that replica.
    • If all replicas are lagging behind (min_lsn), the proxy either waits briefly (< 20ms) or falls back to querying the Primary database.

2. Time-Based Primary Sticky Pinning (Heuristic Approach)

  • Whenever a user performs a write operation, set an ephemeral session flag (in Redis or an encrypted cookie) with a TTL equal to 3 × p99 replication lag (e.g., 5 seconds).
  • For the next 5 seconds, all read queries from that specific user session are pinned directly to the Primary Database.
  • After 5 seconds, reads smoothly revert back to the Read Replica pool.

3. User-ID Affinity Routing

  • Partition users consistently across replica subsets using consistent hashing on (hash(user_id)). If writes to that user's partition stream to a dedicated local replica, the client always reads from the replica receiving their stream.

03.3. Monotonic Reads & Monotonic Writes Guarantees

Read-Your-Writes is part of the four classical Client-Centric Consistency Guarantees:

  1. Read-Your-Writes: A user's successive reads will always reflect their own previous writes.
  2. Monotonic Reads: If a user reads version v_2 of a record, all subsequent reads by that user will return version v_2 or newer—they will never "time-travel" backwards to observe an older version v_1 (which happens if successive reads hit differently lagging replicas).
  3. Monotonic Writes: A user's successive write operations are guaranteed to be serialized and executed in the exact order they were submitted.
  4. Writes-Follow-Reads (Causal Writes): If a user writes an update after reading version v_1, that new update is guaranteed to causally take effect after v_1.

⚖️Architectural Trade-offs & Production Realities

Architectural Advantages

  • Completely eliminates user confusion and duplicate form submissions caused by replication lag
  • Preserves 90%+ offloading of read traffic to horizontal read replicas for browsing users

Trade-offs & Constraints

  • Adds routing state metadata in cookies/headers and requires database proxy or gateway awareness
  • Heavy-writing users will frequently route reads to the primary database, slightly reducing replica offloading
Production Implementation in Big Tech
Facebook (Meta)• TAO Cache Read-After-Write

When a user posts a status on Facebook, TAO routes subsequent reads from that specific user to their local leader master cache to guarantee immediate visibility.

🎯 Staff+ Engineering Takeaways

  • Read-Your-Writes guarantees a client always sees their own recent updates.
  • Achievable via LSN cookies, read-after-write primary pinning, or monotonic session tokens.
  • Monotonic reads prevent time-travel anomalies where subsequent requests hit lagging replicas.
  • Protects user trust without needing global distributed locks.

Topic Knowledge Assessment 🧠

Step through 1 scenario question to test your staff-level grasp.

Question 1 of 10 answered
#1

How can an API Gateway enforce Read-Your-Writes without routing all reads to the primary database?

Rate This Architecture Chapter4.9 / 5.0 (38 ratings)

How clear and staff-actionable was this system breakdown?