# MySQL > 6 posts about MySQL, summarised, each linking to the original. ## Articles ### [The Hugo evolution: Engineering Grab's unified, one-click data ingestion platform with Apache Flink](https://yomu.fyi/post/the-hugo-evolution-engineering-grab-s-unified-one-click-data-ingestion.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Shuguang Xiang - Published: May 22, 2026 Grab's self-service data platform, Hugo, faced significant onboarding friction as streaming pipelines expanded across fragmented systems like Kafka Connect, custom Go applications, and Spark. Engineering teams struggled with cross-platform configuration translations and brittle, manual schema mappings that stretched onboarding over several days. To resolve these bottlenecks, Grab modernized the ingestion architecture by introducing a centralized automation layer powered by Apache Flink and Flink CDC. The updated platform dynamically retrieves Protobuf schemas from Confluent Schema Registry and ingests MySQL binlogs directly into queryable Hive tables without intermediate Kafka hops. This shift dropped pipeline onboarding times to roughly six minutes for Kafka and three minutes for MySQL CDC, driving more pipeline adoptions in one year than in the previous five. ### [How we store and process millions of orders daily](https://yomu.fyi/post/how-we-store-and-process-millions-of-orders-daily.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Xi Chen - Published: Aug 15, 2022 The Grab Order Platform processes millions of food and mart transactions daily, requiring high throughput, fault tolerance, and reduced cloud costs across transactional and analytical workloads. To meet these demands, the engineering team decoupled their database architecture by using Amazon DynamoDB for critical OLTP queries and MySQL RDS for historical OLAP queries. DynamoDB handles online order lifecycles with strong consistency, utilizing sparse Global Secondary Indexes for ongoing orders and TTL configurations to limit storage growth. Updates propagate asynchronously to MySQL RDS through a Kafka ingestion pipeline backed by Amazon SQS retries and timestamp-based version checks. This dual-database approach isolated core transaction availability from analytical queries and delivered significant cloud cost savings. ### [How Kafka Connect helps move data seamlessly](https://yomu.fyi/post/how-kafka-connect-helps-move-data-seamlessly.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Wenli Wan - Published: Apr 6, 2022 Grab's real-time data platform team, Coban, implemented a managed Kafka Connect ecosystem on Kubernetes to streamline moving data in and out of Apache Kafka. To resolve dual-write consistency issues and capture pre- and post-change data, the team integrated Debezium connectors to capture MySQL binlog events and accommodate database DDL migrations. For disaster recovery and stream migrations, Coban deployed MirrorMaker2 connectors managed via Terraform to handle message mirroring and consumer offset translation across AWS regions. Additionally, they developed a custom converter utilizing Confluent Schema Registry to transform Protobuf-serialized Kafka records into JSON for ingestion into Azure Event Hubs. This architecture enabled zero-downtime cluster migrations and robust cross-region disaster recovery. ### [Real-time data ingestion in Grab](https://yomu.fyi/post/real-time-data-ingestion-in-grab.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Shuguang Xiang - Published: Mar 14, 2022 Service teams at Grab historically had to dual-write transactional data into databases and Kafka, creating data integrity issues during transaction failures alongside substantial schema maintenance overhead. To overcome these limitations and eliminate burst reads from SQL-based queries, the Caspian team built a real-time ingestion platform synchronising MySQL, Aurora, and DynamoDB directly to Kafka. For MySQL and Aurora, the platform uses Debezium with Kafka Connect on ROW-format binlogs, while DynamoDB changes are captured via DynamoDB streams with auto-scaling AWS Lambda functions. Messages encoded in Protobuf are transported via Kafka and ingested into Amazon S3 using a Golang stream processor. This architecture supports search indexing in Elasticsearch, automated data lake pipelines, cross-region disaster recovery replication, and audit trails. ### [Data First, SLA Always](https://yomu.fyi/post/data-first-sla-always.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Johan Kok - Published: Aug 1, 2019 Grab's Data Engineering team transitioned from periodic batch ETL ingestion to a real-time change data capture architecture called Trailblazer after dataset sizes exceeded the petabyte mark. The previous system caused severe JDBC timeouts and heavy CPU loads when executing chunked or full-scan queries on unindexed upstream MySQL tables. To solve this, MySQL binary logs are captured via Debezium on Kafka Connect, buffered in Kafka, and ingested into a data lake using Spark Structured Streaming. Checkpoints are decoupled from local storage and persisted in a Redis cluster to simplify ingestion offset overrides and handle ephemeral compute clusters. The system incorporates extensive health monitoring across Airflow, Datadog, and custom services to maintain stream liveliness and avoid Kafka retention breaches. ### [Deep Dive into Database Timeouts in Rails](https://yomu.fyi/post/deep-dive-into-database-timeouts-in-rails.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Jia Hao Goh - Published: Jan 29, 2018 Following a production outage where a database failover caused a Ruby on Rails application to exhaust its Puma server threads, an investigation was conducted to understand how ActiveRecord and MySQL timeout settings behave. A reproduction environment using Docker, Puma, and Toxiproxy replicated how hanging requests to a failing database consume all available server threads, ultimately starving unrelated endpoints. The analysis breaks down ActiveRecord connection pooling mechanics alongside underlying mysql2 and libmysqlclient settings, specifically checkout\_timeout, connect\_timeout, and read\_timeout. Testing confirmed how existing and new TCP connections transition through socket states during network interruptions while waiting on configured timeout intervals.