The SQS Decision Most Diagrams Skip: Normal Work vs. Broken Work
Amazon SQS connects producers and workers, but its real design test is failure. Compare the processing path with the signals and controls needed when queue operations break.
Loading…
Topic
Amazon SQS
2 posts about Amazon SQS. Every summary links to the original.
- 2 posts
- 1 company
- 1 explainer
See the idea
Amazon SQS explainers
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Posts
Building resilient real-time streaming workers with Amazon DynamoDB leases
Managing hundreds of persistent outbound WebSocket connections requires explicit ownership because a failed worker can drop connections and stop data flow until they are reassigned. The described fleet-management pattern uses Amazon DynamoDB leases, conditional writes, and a global secondary index to ensure one worker owns each connection, renew ownership with heartbeats, and find expired leases for reconciliation. Amazon ECS on AWS Fargate workers poll Amazon SQS for notifications, maintain the WebSockets, and release or transfer ownership during graceful shutdown, failures, and scaling. The design also records desired state, the upstream URL, and the last processed sequence number in the lease item, while clock synchronization supports its time-based expiry. The post identifies heartbeat writes as the dominant DynamoDB cost and recommends provisioned capacity with Auto Scaling for sustained high connection counts, while noting that longer intervals reduce cost but extend failover.
Testing application resilience with Amazon SQS and AWS Fault Injection Service
The post presents a resilience-testing tutorial for applications that depend on Amazon Simple Queue Service (Amazon SQS), where failed queue operations can stall downstream processing and trigger cascading outages. It uses AWS Fault Injection Service (AWS FIS) and AWS Systems Manager (SSM) Automation to apply a scoped deny resource policy that blocks data-plane actions, including sending, receiving, deleting, changing visibility, and purging, while preserving queue management. The experiment increases disruption across four phases, then observes producer-side and consumer-side behavior through application and Amazon CloudWatch metrics, including circuit-breaker state, failed sends, backlog, message age, duplicates, and dead-letter-queue effects. The guidance emphasizes hypotheses with measurable success criteria, non-production guardrails, tagged target queues, and a stop condition based on customer impact. It also describes consumer-only, partial, and compound-failure extensions, warning that concurrent experiments on one queue can leave a stale deny policy.
Related topics
All topics →Weekly digest
The best engineering reads, once a week.