Loading…
Grab
Southeast Asia's leading everyday "super-app" that provides ride-hailing, food delivery, grocery shopping, and digital financial services.
- 242 posts summarised
- latest
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
From their blog
Latest articles
A Lean and Scalable Data Pipeline to Capture Large Scale Events and Support Experimentation Platform
Controlled online experimentation across diverse product verticals requires tracking interactions across systems to prevent local optimizations from causing global degradation. Grab built a batch data pipeline to capture, ingest, and process petabytes of event data to support its experimentation platform and analytics stakeholders. The architecture loads ingested event data from Amazon S3, transforms and sorts it, and writes partitioned output back to S3 with metadata registered in Apache Hive. Using Apache Spark on AWS Elastic MapReduce with Apache Airflow for orchestration, the system handles roughly 400,000 incoming events per second. The data is partitioned by event type and ingestion time and stored in Apache ORC format to streamline query workloads and reduce retrieval overhead.
Designing Resilient Systems: Circuit Breakers or Retries? (Part 2)
Retries enable software systems to recover from transient upstream failures by automatically repeating unsuccessful requests. While retrying increases the chance of request completion across multi-host setups, it consumes additional CPU and time without inherently tracking host health. Applications must selectively retry errors with a likelihood of success, such as 500 and 503 status codes, while avoiding client-side failures like 400 or 401. To manage distributed systems safely, retries require idempotent operations or cryptographic nonces, along with backoff and jitter to prevent request stampedes. Tuning retry counts, timeouts, and delays is critical to cap the worst-case consumer response time.
Querying Big Data in Real-time with Presto & Grab's TalariaDB
Grab developed TalariaDB to support real-time SQL querying over high-velocity event streams while maintaining predictable sub-second latencies and low infrastructure costs. The distributed time-series store retains only the most recent hour of data and integrates directly with Presto via its PrestoThriftService interface. Internally, TalariaDB uses the Go-based Badger key-value store to maintain an in-memory key index of metric names and timestamps while mapping columnar event payloads directly to disk. Ingestion occurs by processing pre-partitioned event batches written to Amazon S3 via SQS notifications. By combining a zero-copy decoder with parallel split evaluation across gossiping cluster nodes, the architecture scales horizontally while serving millions of events per second.
Designing Resilient Systems: Circuit Breakers or Retries? (Part 1)
Distributed architectures frequently encounter upstream failures triggered by networking issues, system overloads, resource starvation, and invalid deployments. Implementing software circuit breakers interposes a monitoring mechanism between components to halt requests when failure thresholds are met, giving struggling upstream dependencies time to recover. Circuit breakers save CPU, memory, and network resources by failing fast or routing execution through defined fallbacks such as cached data, alternate services, or approximation algorithms. Grab utilizes Hystrix-Go to manage upstream interactions and configure key thresholds for concurrency, timeouts, and error ratios. This approach protects downstream consumers from cascading latency while insulating upstream resources from excess traffic.
Orchestrating Chaos Using Grab's Experimentation Platform
Grab operates hundreds of microservices where failures in non-critical components can cause outages in critical user flows if fallback mechanisms are improperly configured. To validate system resilience, Grab built Chaos ExP by layering a chaos engineering SDK and dedicated web UI on top of its existing Experimentation Platform. Integrated directly into the Grab-Kit server middleware, the framework intercepts incoming requests and evaluates whether to inject failures using local variable resolution. Supported failure primitives include latency, errors, panics, rate throttling, and resource leaks to test dependent services. Combining chaos testing with experimentation telemetry enables engineers to correlate injected infrastructure disruptions with business metric impacts.
Reliable and Scalable Feature Toggles and A/B Testing SDK at Grab
Grab previously managed experiments using custom service-level code and a toggling library that queried a shared Redis instance, creating latency risks and a single point of failure across backend microservices. To achieve reliable, sub-microsecond feature evaluations, the team designed a Go SDK that resolves rollouts and A/B tests entirely in memory without runtime network I/O. Backend services periodically poll JSON-defined configuration schemas stored in Amazon S3 through a Universal Configuration Manager. The SDK evaluates contextual attributes called facets locally and pushes decision telemetry asynchronously to an S3 and Presto data lake. This architecture allows engineering and product teams to gate deployments and run server-side experiments safely without service disruption.
Mockers - Overcoming Testing Challenges at Grab
Grab operates over 250 microservices communicating over HTTP and gRPC, making shared staging environments costly, ambiguous in ownership, and fragile due to inconsistent data and uncoordinated deployments. To address these testing bottlenecks, Grab created Mockers, a Go SDK and CLI tool backed by a central monorepo of mock servers for local-box and CI testing. Mockers automatically generates HTTP and gRPC mock servers from Swagger specifications and protobuf files, returning configured network responses without internal business logic. By incorporating Grab's in-house chaos SDK middleware, Mockers also enables repeatable resiliency and contract testing locally without relying on code-level mocks. While Grab still mandates integration testing on distributed staging environments with live data, Mockers enables developers to detect complex defects and contract mismatches earlier.
Journey of a Tourist via Grab
Grab analyzed platform ride data from millions of tourist passengers representing over 150 countries visiting Singapore. Over 60% of these tourist riders originated from Southeast Asia, while non-regional visitors mainly came from China, the United States, and India. Seasonal demand revealed a trimodal distribution for tropical travelers aligning with holiday periods, contrasting with a September-to-January peak for visitors escaping winter in four-season climates. Airport trips showed that nearly 90% of tourist passengers headed directly to hotels, concentrated heavily in central areas like Orchard, Bugis, Downtown Core, and Kallang. Additional key destinations included major shopping districts, iconic dining locations like Newton Food Centre and Chijmes, and medical centers, which saw tourist ride volume grow over 500% between 2015 and 2017.
How We Designed the Quotas Microservice to Prevent Resource Abuse
As Grab migrated from a monolith to hundreds of microservices, managing global rate limiting became essential to prevent cascading failures and resource exhaustion. To avoid putting a rate limiting service on the critical path of every API call, Grab built Quotas, an asynchronous rate limiting system. Client services use a lightweight SDK and middleware to read rate limiting decisions from local in-memory caches and stream usage metrics asynchronously via Apache Kafka. The Quotas service aggregates usage data locally, flushes stats to Redis periodically, and publishes updated rate limiting decisions back over Kafka topics. In production, Quotas successfully handles 200k peak transactions per second with decision enforcement delays capped at 200 milliseconds.
Grab Senior Data Scientist Liuqin Yang Wins Beale-Orchard-Hays Prize
Grab Senior Data Scientist Dr. Liuqin Yang, Professor Defeng Sun, and Professor Kim-Chuan Toh received the 2018 Beale-Orchard-Hays Prize for their research paper introducing SDPNAL+. The software employs a majorised semismooth Newton-CG augmented Lagrangian method to solve large-scale semidefinite programming problems with nonnegative constraints. While traditional methods struggled beyond matrix dimensions of 2,000 and 5,000 constraints, SDPNAL+ successfully scales to matrix dimensions of 9,261 and over 12 million constraints. In benchmark testing, the software solved a problem on a desktop PC in 1.5 hours that required 122 hours on a 56-core CPU and 128-GPU cluster using a traditional solver. Grab implements these optimisation techniques to accelerate its passenger-driver allocation algorithms by hundreds of times.
Building Grab’s Experimentation Platform
Grab built its internal Experimentation Platform (ExP) to replace a manual, expensive testing process that required bespoke meetings, custom logging pipelines, and service modifications for each experiment. ExP provides a unified infrastructure featuring a centralized management UI, automated real-time data streaming to S3, and SDKs for Android, iOS, and Go. The platform leverages JSON-based experiment definitions delivered through dynamic configuration management, enabling client-side evaluation without costly network calls. It addresses marketplace network effects and inter-experiment interference through mechanisms such as geo-temporal segmentation and domain-layer models. The platform has scaled to run approximately 25 concurrent experiments while computing roughly 2,500 metrics and 50,000 experiment-metric combinations daily.
Introducing Grab-Kit: Distributed Service Design at Grab
As Grab migrated from a monolith to microservices, maintaining consistency, coordination, and code quality across rapidly expanding teams became a major engineering challenge. To address this, the Developer Experience team built Grab-Kit, a Go framework that automates service scaffolding, code generation, and distributed system design patterns. The framework uses Protocol Buffer definition files as a single source of truth to generate data transfer objects, communication bindings, and standardized middleware for logging and profiling. Grab-Kit also features declarative metrics definitions that synchronize with the DataDog API to build and update service dashboards automatically. Adopting the framework reduced development time for creating new services by up to 70% in teams such as GrabFood while improving overall system stability.
How Grab Experimented with Chat to Drive Down Booking Cancellations
Post-allocation ride cancellations at Grab degrade the booking experience and create costly inefficiencies for both passengers and driver-partners. Internal user research and platform data confirmed that rides involving GrabChat conversations had significantly lower cancellation rates by reducing perceived wait times. To scale this interaction without extra cost, the team tested system-generated automated messages sent at varying delay intervals, styles, tones, and localized verbiage across different cities. Faster message delivery outperformed longer delays, and tailored prompts reduced booking cancellations by up to two percentage points across tested markets. The experiment demonstrated that high-quality, directed prompts solicited quick responses and improved pick-up efficiency even when overall message volume was lower than control groups.
Deep Dive into Database Timeouts in Rails
Following a production outage where a database failover caused a Ruby on Rails application to exhaust its Puma server threads, an investigation was conducted to understand how ActiveRecord and MySQL timeout settings behave. A reproduction environment using Docker, Puma, and Toxiproxy replicated how hanging requests to a failing database consume all available server threads, ultimately starving unrelated endpoints. The analysis breaks down ActiveRecord connection pooling mechanics alongside underlying mysql2 and libmysqlclient settings, specifically checkout_timeout, connect_timeout, and read_timeout. Testing confirmed how existing and new TCP connections transition through socket states during network interruptions while waiting on configured timeout intervals.
Dealing with the Meltdown Patch at Grab
AWS infrastructure maintenance related to Meltdown patches led to severe CPU utilization spikes across Grab's ElastiCache Redis instances. Because Redis is single-threaded, spikes past 50% CPU on two-vCPU instances threatened service capacity, and initial Multi-AZ failovers only provided temporary relief until the new master nodes received rolling patches. To handle the increased overhead before their peak traffic window, the engineering team horizontally scaled both clustered and non-clustered Redis fleets. For Redis 3.2.4 clusters lacking live re-sharding support, they provisioned larger clusters, warmed caches, and redirected traffic. Non-clustered workloads were resolved by provisioning extra nodes, migrating compatible services to Redis Cluster, or updating application code to shard data across multiple instances.
GrabShare at the Intelligent Transportation Engineering Conference
Grab presented a technical paper on the construction of its real-time ridesharing service, GrabShare, at the Intelligent Transportation Engineering Conference in Singapore. The platform pairs passengers heading along similar routes with drivers immediately while handling network drops, volatile supply and demand, and heavy traffic conditions in Southeast Asian cities. To deliver accurate pairings, the scheduling system generates and filters through hundreds of travel time estimates for each candidate match before finalizing an itinerary. Operational teams on the ground evaluate complaints about poor matches, enabling engineers to refine the online matching systems. Over the course of one month, the service cut more than 4.5 million kilometers of driving distance and brought in over 100,000 new users within two weeks.
Grabbing Growth: A Growth Hacking Story
Grab established a dedicated Growth Hacking team within its Technology organization to pursue high-risk, niche initiatives and scale impact across 68 million regional users. Positioning the team within engineering allowed rapid A/B testing and simultaneous multi-market deployments driven by structured growth loops. The team prioritizes and evaluates all initiatives using a Growth Factor metric, calculated as the increase in rides divided by the increase in costs. To improve driver engagement, the team implemented a Spin-to-Win game based on B.F. Skinner's variable ratio reinforcement principles, delivering probabilistic monetary and merchandise rewards upon meeting daily ride thresholds. Ongoing regional experiments continue to evaluate metrics including driver acceptance, cancellation rates, and driver ratings.
The Data and Science Behind GrabShare Part I: Verifying Potential and Developing the Algorithm
Expanding from point-to-point dispatch services to dynamic carpooling requires matching independent passenger requests traveling in similar directions without causing unacceptable delays. Grab evaluated the feasibility of its GrabShare service by analyzing historical trip data with DBSCAN clustering on coordinates projected into a Universal Transverse Mercator system. This analysis demonstrated that 35% to 46% of rides across typical daytime windows fell into tight geographic clusters with near-identical pickup and drop-off coordinates. The resulting matching framework adapts the baseline dispatch flow by searching for in-transit drivers and enforcing real-time seat reservation constraints. Route assignment decisions subsequently evaluate detour times, trip angles, and expected arrival times to ensure driver utilization improves while total driving distance decreases.
The Art of Hiring Good Engineers
Building and scaling engineering teams requires adapting recruitment strategies across different organizational sizes, from initial hires to hundreds of engineers. Teams often begin with small groups of battle-tested engineers before designing formal hiring pipelines. The recruitment approach outlines four key steps: sourcing top-tier talent from platforms like GitHub and LinkedIn, assessing technical fit through tailored screens or assignments, evaluating cross-functional and culture fit via group interviews, and optimizing the process. This optimization includes routing candidates strategically to strict bar-raiser interviewers to prevent burnout and expedite decisions.
Migrating Existing Datastores
Grab's Identity team faced imminent memory exhaustion on a single Redis node used to cache mobile authentication tokens under rapid user growth. Because read traffic outweighed write traffic by roughly 200 times, the team opted for an AWS ElastiCache cluster with three shards and two read replicas per shard. They executed a zero-downtime, six-phase migration plan while handling a peak load of 20,000 queries per second. The migration transitioned through initial one-time data replication, asynchronous shadow writes, synchronous dual writes, asynchronous read validation, switching primary reads, and final write cleanup. Controlled by feature flags and monitored with metrics at every stage, the migration completed without invalidating tokens or causing service disruptions.
Topics they write about
All topics →Weekly digest
The best engineering reads, once a week.