Stateless vs Stateful Services: Architecture & Elasticity

The distinction between stateless and stateful services centers on where mutable state resides during request lifecycles:

• Stateless Services: Treat each incoming request as an independent, isolated transaction. The server holds zero long-term client context in process memory or local ephemeral disk. Any state required to process a request is either passed within the request itself (e.g., cryptographically signed JWT claims) or fetched dynamically from shared external systems (e.g., Redis, Cassandra, PostgreSQL, S3).
• Stateful Services: Retain client conversation state, transactional buffers, open TCP socket descriptors, or local database page caches in local memory or persistent attached disks across multiple sequential requests.

The Sticky Session Anti-Pattern

When legacy monolithic applications store user sessions in local server RAM (e.g., HttpSession in Java or memory sessions in Express.js), the load balancer is forced to enable Session Affinity (Sticky Sessions) via HTTP cookies or client IP hashing:

1. Traffic Skew: A small fraction of power users generate 80\% of traffic, overloading specific backend nodes while others sit idle.
2. Brittle Failover: When a server instance crashes or is rotated during deployment, every active user pinned to that machine suffers immediate session loss, cart drops, and abrupt logout.
3. Impaired Autoscaling: Scaling out new nodes does not relieve loaded servers because existing users remain pinned to legacy nodes.

While web APIs should be stateless, foundational distributed systems (Kafka brokers, Cassandra nodes, Elasticsearch clusters, ZooKeeper/etcd nodes) are intrinsically stateful.

In Kubernetes, managing stateful workloads requires StatefulSets rather than standard Deployments:

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: cassandra-node
spec:
  serviceName: "cassandra-headless"
  replicas: 3
  selector:
    matchLabels:
      app: cassandra
  template:
    metadata:
      labels:
        app: cassandra
    spec:
      containers:
      - name: cassandra
        image: cassandra:4.1
        ports:
        - containerPort: 9042
          name: cql
        volumeMounts:
        - name: cassandra-data
          mountPath: /var/lib/cassandra
  volumeClaimTemplates:
  - metadata:
      name: cassandra-data
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 500Gi

Certain systems are inherently stateful at the transport layer, such as WebSockets, gRPC bidirectional streams, and real-time multiplayer game servers. An open TCP connection locks a file descriptor and memory buffer in a specific host's Linux network stack.

How to Scale Real-Time Stateful Gateways:

1. Stateless Gateway Tier with Pub/Sub Backplane: WebSocket gateway servers accept client connections, but maintain zero application business logic. When User A (connected to Gateway Server 1) sends a chat message to User B (connected to Gateway Server 8), Server 1 publishes the event to Redis Pub/Sub or Apache Kafka. Server 8 subscribes to the channel and pushes the message down User B's active WebSocket.
2. Session Registry / Presence Service: A shared distributed key-value store (e.g., Redis) tracks userId -> gateway_pod_id. Incoming push notifications query the registry to route RPCs to the exact node hosting that user's open TCP socket.
3. Graceful Connection Draining: During deployments, WebSocket servers catch SIGTERM, stop accepting new connections, send a reconnect_after(jitter) frame to active clients, and wait 30–60 seconds before process exit.

import { createClient } from 'redis';
import WebSocket, { WebSocketServer } from 'ws';

const wss = new WebSocketServer({ port: 8080 });
const subClient = createClient({ url: 'redis://redis-cluster:6379' });
const pubClient = createClient({ url: 'redis://redis-cluster:6379' });

const localSockets = new Map<string, WebSocket>();

wss.on('connection', (ws, req) => {
  const userId = extractUserId(req);
  localSockets.set(userId, ws);

  ws.on('message', async (data) => {
    const { targetUserId, message } = JSON.parse(data.toString());
    // Publish to Redis channel so any gateway node holding targetUserId can deliver it
    await pubClient.publish(`user-channel:${targetUserId}`, JSON.stringify({ from: userId, message }));
  });

  ws.on('close', () => {
    localSockets.delete(userId);
  });
});

// Redis subscriber picks up messages routed to users connected to THIS node
await subClient.pSubscribe('user-channel:*', (message, channel) => {
  const targetUserId = channel.split(':')[1];
  const targetSocket = localSockets.get(targetUserId);
  if (targetSocket && targetSocket.readyState === WebSocket.OPEN) {
    targetSocket.send(message);
  }
});

A primary financial superpower of stateless services is the ability to run on AWS Spot Instances / GCP Preemptible VMs, which offer 70\% - 90\% discounts compared to On-Demand pricing.

Handling 2-Minute Preemption Notices

Because cloud providers can reclaim Spot instances with a 2-minute warning when demand spikes:

• Stateless Pods: Receive an AWS EventBridge termination event → Kubernetes drains the node → AWS ALB unregisters the target → traffic seamlessly flows to surviving nodes with zero user impact.
• Stateful Pods on Spot: Highly risky; if a database primary node is terminated before data logs flush or replicas elect a new leader, data corruption or failover latency spikes occur. State-bearing primaries must always run on On-Demand or Reserved capacity.