How we store and process millions of orders daily

Summary

The Grab Order Platform processes millions of food and mart transactions daily, requiring high throughput, fault tolerance, and reduced cloud costs across transactional and analytical workloads. To meet these demands, the engineering team decoupled their database architecture by using Amazon DynamoDB for critical OLTP queries and MySQL RDS for historical OLAP queries. DynamoDB handles online order lifecycles with strong consistency, utilizing sparse Global Secondary Indexes for ongoing orders and TTL configurations to limit storage growth. Updates propagate asynchronously to MySQL RDS through a Kafka ingestion pipeline backed by Amazon SQS retries and timestamp-based version checks. This dual-database approach isolated core transaction availability from analytical queries and delivered significant cloud cost savings.

Takeaways

1. 1

   DynamoDB handles OLTP queries using sparse Global Secondary Indexes that automatically delete entries when order status updates clear the index key, keeping ongoing order lookups compact.

2. 2

   The data ingestion pipeline pairs Kafka with Amazon SQS fallback queues, database unique keys, and microsecond-precision update timestamps to handle retries, deduplication, and out-of-order events.

3. 3

   Data retention policies reduce storage costs by dropping monthly MySQL partitions older than six months and setting DynamoDB TTL expirations to three months.