Load Balancing Algorithms (Round Robin, Least Connections, Hash, Weighted)

Load balancing algorithms determine how incoming requests or network flows are distributed across a pool of available backend servers. They are categorized into two fundamental classes:

1. Static Algorithms: Make routing decisions based purely on mathematical sequences or deterministic hashing, without considering the real-time CPU load, memory utilization, or connection count of backend servers.
2. Dynamic Algorithms: Continuously collect runtime telemetry (active TCP connection counts, response latency histograms, CPU metrics) and adjust routing decisions dynamically in real time.

1. Round Robin & Weighted Round Robin (WRR)

• Round Robin: Sequentially assigns request 1 to Server A, request 2 to Server B, request 3 to Server C, and wraps around.
• Weighted Round Robin (WRR): Assigns integer weights based on server hardware capacity (e.g., a 64-core server gets weight 4; a 16-core server gets weight 1). The algorithm sends 4 requests to the 64-core server for every 1 request sent to the 16-core server.
• Best For: Homogeneous stateless web APIs with uniform, short request processing times.
• Failure Mode: If some requests take 10 seconds (complex reports) and others take 5ms, Round Robin can pile heavy requests onto the same server, triggering CPU overload.

2. Least Connections & Weighted Least Connections (WLC)

• The load balancer tracks the exact number of active, open TCP connections on each backend.
• New requests are dispatched to the server currently handling the fewest active connections.
• Best For: Long-lived connections (WebSockets, database connection pools, file uploads, large video streaming).

3. IP Hash & Consistent Hashing

• Computes a cryptographic hash of the client's IP address or session token: Server Index = hash(Client IP) \pmod N.
• Guarantees that requests from the same user are consistently routed to the exact same backend server (Session Affinity / Sticky Sessions).
• Best For: In-memory user sessions, stateful local caching, chunked file upload pipelines.

4. Least Response Time & Peak EWMA (Exponentially Weighted Moving Average)

• Combines the number of active connections with the server's rolling average latency:

Score = Active Connections × EWMA Latency

• The load balancer routes traffic to the server that produces the lowest score.
• Best For: Microservices with heterogeneous workloads (used heavily by Twitter/X Finagle and Envoy Proxy).

5. Google Maglev Hashing

• A consistent hashing algorithm developed by Google that ensures deterministic lookup in O(1) time while minimizing key reshuffling when backends crash or scale out, maintaining consistent packet flows across Line-Rate L4 switches.

# Least Connections Algorithm
upstream dynamic_backend {
    least_conn;
    server 10.0.1.11:8080 weight=3;
    server 10.0.1.12:8080 weight=1;
}

# IP Hash for Session Stickiness
upstream sticky_backend {
    ip_hash;
    server 10.0.1.11:8080;
    server 10.0.1.12:8080;
}