---
title: "DynamoDB"
description: "10 posts about DynamoDB, summarised, each linking to the original."
---

# DynamoDB
> 10 posts about DynamoDB, summarised, each linking to the original.

## Articles

### [Multi-region disaster recovery for Amazon API Gateway WebSocket APIs](https://yomu.fyi/post/multi-region-disaster-recovery-for-amazon-api-gateway-websocket-apis.md)
- Company: [AWS](https://yomu.fyi/company/aws.md)
- Author: Shubham Dhoka
- Published: Oct 14, 2026
- Type: Tutorial

WebSocket connections for real-time applications like live auctions and collaborative editors are vulnerable to regional outages because each connection is tied to a single AWS Region. A regional failure severs all active connections and loses session state. This post presents an active-passive multi-region disaster recovery pattern that uses Amazon API Gateway WebSocket APIs, Route 53 failover routing, DynamoDB Global Tables for cross-Region state replication, a canary health probe, and client-side reconnection logic. The canary Lambda function tests the endpoint every minute and drives a CloudWatch alarm that triggers a Route 53 health check to mark the primary Region unhealthy. Clients detect connection drops via application-level heartbeat messages and reconnect with exponential backoff and full jitter to avoid thundering herd. After reconnection, clients retrieve the latest application state from the secondary Region’s DynamoDB replica and resume their session automatically.


### [Bulk Executor for Amazon DynamoDB, Part 2: Lessons we learned the hard way](https://yomu.fyi/post/bulk-executor-for-amazon-dynamodb-part-2-lessons-we-learned-the-hard-w.md)
- Company: [AWS](https://yomu.fyi/company/aws.md)
- Author: Jason Hunter
- Published: Oct 13, 2026
- Type: Problem & solution

The post describes design tradeoffs and workarounds encountered while building the open-source Bulk Executor tool for Amazon DynamoDB on AWS Glue. Core requirements included real-time local-script feel, scaling to hundreds of parallel workers without user infrastructure management, and easy command extensibility. AWS Glue was chosen for serverless parallelism and a built-in DynamoDB connector, but the connector simplified data types, losing distinctions like string set versus list of strings. Challenges included estimating costs for provisioned tables, handling job termination with ^C, and streaming output via CloudWatch Live Tail which samples events and delivers out-of-order. Solutions included a two-phase cost estimate for deletes, programmatic interrupt handling, writing large output to S3 with a preview, mandating point-in-time recovery for mutating commands, and using AI for testing and linting; the team also migrated from AWS Glue 4 to 5 and 6 for faster startup and 30 percent lower DPU costs.


### [Building resilient real-time streaming workers with Amazon DynamoDB leases](https://yomu.fyi/post/building-resilient-real-time-streaming-workers-with-amazon-dynamodb-le.md)
- Company: [AWS](https://yomu.fyi/company/aws.md)
- Author: Siddhesh Tiwari
- Published: Sep 10, 2026
- Type: Tutorial

Managing hundreds of persistent outbound WebSocket connections requires explicit ownership because a failed worker can drop connections and stop data flow until they are reassigned. The described fleet-management pattern uses Amazon DynamoDB leases, conditional writes, and a global secondary index to ensure one worker owns each connection, renew ownership with heartbeats, and find expired leases for reconciliation. Amazon ECS on AWS Fargate workers poll Amazon SQS for notifications, maintain the WebSockets, and release or transfer ownership during graceful shutdown, failures, and scaling. The design also records desired state, the upstream URL, and the last processed sequence number in the lease item, while clock synchronization supports its time-based expiry. The post identifies heartbeat writes as the dominant DynamoDB cost and recommends provisioned capacity with Auto Scaling for sustained high connection counts, while noting that longer intervals reduce cost but extend failover.


### [How We Used AI to Bring MongoDB to DynamoDB](https://yomu.fyi/post/how-we-used-ai-to-bring-mongodb-to-dynamodb.md)
- Company: [MongoDB](https://yomu.fyi/company/mongodb.md)
- Published: Aug 24, 2026
- Type: Announcement

AWS released ExtendDB, an open source adapter that lets DynamoDB workloads use pluggable storage backends, enabling them to run outside AWS. MongoDB built a storage backend for ExtendDB, allowing DynamoDB workloads to run on MongoDB anywhere, including on-premises and air-gapped environments, without changing application code. The backend maps DynamoDB concepts like partition keys, secondary indexes, transactions, and streams to MongoDB primitives, with tradeoffs such as handling numeric precision differences in the Rust layer. The project began as a hackathon experiment where an engineer used Claude Code to prototype the backend, which quickly produced a working component, then required several weeks of partnering with AWS to merge. The post emphasizes that AI changed the economics of such experiments, enabling the idea to move from thought to implementation quickly, and the backend is now available as open source.


### [Zero-Sum by Design: 10 Years of Uber’s Payments Platform](https://yomu.fyi/post/zero-sum-by-design-10-years-of-uber-s-payments-platform.md)
- Company: [Uber](https://yomu.fyi/company/uber.md)
- Published: Aug 10, 2026
- Type: Explainer

Uber’s Payments Platform, internally called Gulfstream, grew from a small 2016 microservices effort into infrastructure supporting $217 billion in annualized gross bookings, nearly twice that amount in total money movement, and ledger balances for more than 1.2 billion entities. It is built around immutable Money Orders, zero-sum double-entry accounting, strongly consistent ledger balances, and three core models: money orders, ledgers, and entity changelogs. Independent stateless services communicate primarily through Apache Kafka, while generic abstractions for lines of business and payment instruments let new products and methods plug into the platform; synchronous in-session flows use Cadence. As scale and audit requirements increased, Uber introduced Entity Fares, addressed hot ledger entities, and built LedgerStore for auditability and tamper-evident record-keeping, while also extending the ledger as a multi-tenant service.


### [How Vercel used Cursor to build Queues](https://yomu.fyi/post/how-vercel-used-cursor-to-build-queues.md)
- Company: [Cursor](https://yomu.fyi/company/cursor.md)
- Author: Cursor
- Published: Jul 28, 2026
- Type: Explainer

Vercel built its durable event streaming system, Vercel Queues, entirely with Cursor, using agents across three rewrites and validating changes against real AWS environments. Engineers had agents work on the Queues API, connect to AWS, inspect DynamoDB and Kinesis side effects, inject faults, simulate bugs, and verify results, while Cursor retained context through long sessions and compaction events. The workflow also supported model switching, parallel planning agents, inline review, stacked PRs, and asynchronous Cloud Agents. Among Vercel’s Cursor power users, PR throughput increased 54%, PR cycle time decreased 89%, resources per feature fell 27%, and the company reports 104 hours saved per developer annually. Product managers also use Cursor for repository analysis, product analytics, and small fixes without drawing on engineering resources.


### [Two years of vector search at Notion: 10x scale, 1/10th cost](https://yomu.fyi/post/two-years-of-vector-search-at-notion-10x-scale-1-10th-cost.md)
- Company: [Notion](https://yomu.fyi/company/notion.md)
- Author: Preeti Gondi, Mickey Liu, Nathan Louie, Calder Lund, Jacob Sager
- Published: Feb 19, 2026
- Type: Explainer

Notion describes two years of scaling vector search for AI Q&A, where semantic embeddings retrieve workspace and connected-tool content by meaning rather than exact keywords. The initial system combined Apache Spark batch indexing with Kafka-driven real-time updates, workspace-based sharding, and generation routing; rapid demand drove a 600x increase in daily onboarding capacity and cleared a waitlist of millions of workspaces by April 2024. A serverless migration cut vector database costs 50 percent from peak usage, while a later move to turbopuffer reduced search-engine spend 60 percent, lowered AWS EMR costs 35 percent, and improved p50 query latency from 70–100ms to 50–70ms. Further work uses xxHash-based page-state caching in DynamoDB to reprocess only changed spans and shifts embeddings generation from Spark and query embeddings away from a third-party API to Ray on Anyscale, with early rollout results pointing to a projected 90+ percent reduction in embeddings infrastructure costs.


### [Engineering stories behind the Medium Daily Digest Algorithm: Part 2](https://yomu.fyi/post/engineering-stories-behind-the-medium-daily-digest-algorithm-part-2.md)
- Company: [Medium](https://yomu.fyi/company/medium.md)
- Author: Raphael Montaud
- Published: Aug 28, 2025
- Type: Problem & solution

Medium’s Daily Digest filtering had become costly and difficult to maintain after a single Bloom Filter instance accumulated billions of items for read, muted, and presentation rules. The system could face up to 15,000 filter questions per feed and more than 15 million per second across thousands of feeds, while Bloom Filters could not delete or enumerate entries and introduced derived-data drift. The team replaced the muted-writer filter with a user-based database query that fetched all muted writers for the current user and cross-referenced them with the story shortlist, then split read filtering between early user-based queries and later pairwise lookups. The resulting implementation removed Bloom Filters entirely, was reported as ten times cheaper in their situation, made costs independent of shortlist size, supported reading-history clearing, and was easier to control and debug.


### [How we seamlessly migrated high volume real-time streaming traffic from one service to another with zero data loss and duplication](https://yomu.fyi/post/how-we-seamlessly-migrated-high-volume-real-time-streaming-traffic-fro.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Md Riyadh
- Published: Dec 5, 2024
- Type: Problem & solution

Grab migrated its stream-processing (write) functionality from an existing service into a separate service while handling 20,000 reads per second from 16 Kafka streams and writing to output streams and DynamoDB tables. The migration required gradual stream-by-stream rollout, production validation, and a no-downtime switchover; feature flags were rejected because deployment timing could cause loss, duplication, or inconsistent values. The team moved processing into shared /commons code, created temporary validation sinks, and scheduled each switchover with a shared map\[string\]time.Time, allowing both services to change writers at the same time. During each cutover, both services consumed streams, the old service stopped writing to resources, and monitoring continued before old consumption was removed; the result was zero data loss or duplication, no downtime, and completion in about three weeks. It was temporary and tested, but rolling back either service could create inconsistency if systems fell out of sync.


### [Evolving the Backend Storage for Platform Metrics](https://yomu.fyi/post/evolving-the-backend-storage-for-platform-metrics.md)
- Company: [Heroku](https://yomu.fyi/company/heroku.md)
- Author: David Murray
- Published: May 9, 2024
- Type: Problem & solution

Heroku migrated MetaaS, its internal Metrics as a Service platform metrics backend, from self-hosted Apache Cassandra to AWS’s DynamoDB. MetaaS collects application observations, aggregates them into per-application, per-minute statistics such as median, maximum, and 99th-percentile response time, and stores time-series data for dashboard metrics, alerting, and autoscaling. The migration first duplicated Kafka stream-processing writes into DynamoDB, then used “Science Mode” to compare Cassandra and DynamoDB read results in production without changing returned data. After edge cases were fixed and results matched, retention-period TTL expiry avoided a separate historical data transfer, while gradual query cutovers provided another safeguard. Since 100% of MetaaS queries moved to DynamoDB in May of last year, DynamoDB has remained reliable at scale and been faster and cheaper than the Cassandra clusters, with no noticeable effect on users.