Multi-region disaster recovery for Amazon API Gateway WebSocket APIs

Summary

WebSocket connections for real-time applications like live auctions and collaborative editors are vulnerable to regional outages because each connection is tied to a single AWS Region. A regional failure severs all active connections and loses session state. This post presents an active-passive multi-region disaster recovery pattern that uses Amazon API Gateway WebSocket APIs, Route 53 failover routing, DynamoDB Global Tables for cross-Region state replication, a canary health probe, and client-side reconnection logic. The canary Lambda function tests the endpoint every minute and drives a CloudWatch alarm that triggers a Route 53 health check to mark the primary Region unhealthy. Clients detect connection drops via application-level heartbeat messages and reconnect with exponential backoff and full jitter to avoid thundering herd. After reconnection, clients retrieve the latest application state from the secondary Region’s DynamoDB replica and resume their session automatically.

Takeaways

1. 1

   The canary health probe opens a WebSocket connection to the endpoint every minute, sends a heartbeat, and publishes the result as a custom CloudWatch metric to drive failover detection.

2. 2

   Exponential backoff with full jitter prevents a thundering herd during reconnection by randomizing the delay within doubling intervals, spreading retries over time.

3. 3

   State is stored in DynamoDB Global Tables, which replicates asynchronously to the secondary Region typically within a second, but writes immediately before a failure may be lost.

FAQ