Synchronous vs Asynchronous API Design

In distributed systems, operations that take more than 2 to 3 seconds (e.g., video transcoding, PDF invoice generation, machine learning inference, bulk database exports) must never be handled synchronously over a single blocking HTTP request.

Why Long Synchronous Calls Cause Catastrophic Outages:

1. Load Balancer & Gateway Timeouts: Intermediate reverse proxies (AWS ALB, Cloudflare, NGINX) enforce strict timeout limits (typically 30 to 60 seconds). If the backend takes 61 seconds, the gateway terminates the connection with 504 Gateway Timeout, even if the backend continues consuming CPU.
2. Thread & Socket Pool Starvation: Synchronous web servers (Gunicorn, Puma, Tomcat) allocate a worker thread or socket per incoming connection. A surge of 500 concurrent long-running requests blocks all 500 worker threads, causing total system unavailability for fast, lightweight CRUD requests.
3. Ghost Computations: If a mobile client loses cellular connection during a 2-minute synchronous request, the client disconnects, but the backend server blindly completes the expensive computation with no client to receive the result.