---
title: "Building resilient real-time streaming workers with Amazon DynamoDB leases"
description: "Managing hundreds of persistent outbound WebSocket connections requires explicit ownership because a failed worker can drop connections and stop data flow until they are reassigned. The described fleet-management pattern uses Amazon DynamoDB leases, conditional writes, and a global secondary index to ensure one worker owns each connection, renew ownership with heartbeats, and find expired leases for reconciliation. Amazon ECS on AWS Fargate workers poll Amazon SQS for notifications, maintain the WebSockets, and release or transfer ownership during graceful shutdown, failures, and scaling. The design also records desired state, the upstream URL, and the last processed sequence number in the lease item, while clock synchronization supports its time-based expiry. The post identifies heartbeat writes as the dominant DynamoDB cost and recommends provisioned capacity with Auto Scaling for sustained high connection counts, while noting that longer intervals reduce cost but extend failover."
---

# Building resilient real-time streaming workers with Amazon DynamoDB leases

[AWS](https://yomu.fyi/company/aws) · Siddhesh Tiwari · Sep 10, 2026

**Type:** Tutorial

## Summary

Managing hundreds of persistent outbound WebSocket connections requires explicit ownership because a failed worker can drop connections and stop data flow until they are reassigned. The described fleet-management pattern uses Amazon DynamoDB leases, conditional writes, and a global secondary index to ensure one worker owns each connection, renew ownership with heartbeats, and find expired leases for reconciliation. Amazon ECS on AWS Fargate workers poll Amazon SQS for notifications, maintain the WebSockets, and release or transfer ownership during graceful shutdown, failures, and scaling. The design also records desired state, the upstream URL, and the last processed sequence number in the lease item, while clock synchronization supports its time-based expiry. The post identifies heartbeat writes as the dominant DynamoDB cost and recommends provisioned capacity with Auto Scaling for sustained high connection counts, while noting that longer intervals reduce cost but extend failover.

## Context

Long-lived WebSocket connections become unmanaged when workers fail, redeploy, or scale independently. The system requires exactly one worker to own each connection, automatic reassignment after failures, safe draining during deployments, and protection against double-claiming.

## Approach / What changed

Use Amazon DynamoDB items as time-bounded leases, with conditional writes for atomic ownership, heartbeats for renewal, and a global secondary index for finding started connections with expired leases. Amazon ECS on AWS Fargate workers receive notifications through Amazon SQS, reconcile orphaned connections, and use graceful shutdown to release ownership during deployments.

## Takeaways

- DynamoDB conditional writes provide distributed coordination without a separate lock service; a condition on update\_item confirms that one worker still owns a connection before changing its lease.
- A global secondary index on desired\_state and lease\_expires\_at\_ms lets workers query for STARTED connections with expired leases and reconcile connections left unmanaged by failed workers.
- Heartbeat writes dominate DynamoDB cost. Increasing the heartbeat interval lowers WCU consumption but requires a proportionally longer lease, increasing the failover window.

**Tags:** [Amazon SQS](https://yomu.fyi/topic/amazon-sqs), [AWS](https://yomu.fyi/topic/aws), [DynamoDB](https://yomu.fyi/topic/dynamodb), [Python](https://yomu.fyi/topic/python), [WebSockets](https://yomu.fyi/topic/websockets)

- Source: [AWS](https://aws.amazon.com/blogs/architecture/building-resilient-real-time-streaming-workers-with-amazon-dynamodb-leases)
- Source URL: https://aws.amazon.com/blogs/architecture/building-resilient-real-time-streaming-workers-with-amazon-dynamodb-leases
- Ingested by Yomu: 2026-09-10T20:02:44.020Z

[Read original post](https://aws.amazon.com/blogs/architecture/building-resilient-real-time-streaming-workers-with-amazon-dynamodb-leases)