Building Grab’s Experimentation Platform

Summary

Grab built its internal Experimentation Platform (ExP) to replace a manual, expensive testing process that required bespoke meetings, custom logging pipelines, and service modifications for each experiment. ExP provides a unified infrastructure featuring a centralized management UI, automated real-time data streaming to S3, and SDKs for Android, iOS, and Go. The platform leverages JSON-based experiment definitions delivered through dynamic configuration management, enabling client-side evaluation without costly network calls. It addresses marketplace network effects and inter-experiment interference through mechanisms such as geo-temporal segmentation and domain-layer models. The platform has scaled to run approximately 25 concurrent experiments while computing roughly 2,500 metrics and 50,000 experiment-metric combinations daily.

Takeaways

1. 1

   Experiment definitions are structured as formal JSON documents delivered via dynamic configuration management, allowing SDKs to assign treatments deterministically without network calls.

2. 2

   To handle multi-sided marketplace network effects and inter-experiment interference, ExP uses geo-temporal segmentation alongside a mechanism similar to Google's Domains and Layers combined with an expert validation system.

3. 3

   ExP supports diverse assignment strategies including uniform or weighted randomized sampling by user ID and time-sliced allocations with bias minimization for algorithmic testing.