# Grab > Southeast Asia's leading everyday "super-app" that provides ride-hailing, food delivery, grocery shopping, and digital financial services. ## Articles ### [Enabling conversational data discovery with LLMs at Grab](https://yomu.fyi/post/enabling-conversational-data-discovery-with-llms-at-grab.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Shreyas Parbat - Published: Sep 26, 2024 Grab faced major bottlenecks in dataset discovery across its 200,000 data lake tables due to low documentation coverage and vanilla Elasticsearch limitations in its internal tool, Hubble. To resolve this, engineers first tuned Elasticsearch by boosting frequently queried P80 tables and certified schemas while deboosting deprecated datasets. They then deployed a GPT-4 pipeline that generated documentation from table schemas and sample data, raising P80 documentation coverage from 20% to roughly 90%. Finally, the team developed HubbleIQ, an LLM-powered assistant built via Glean Apps and integrated into Slack and Hubble search to answer semantic queries. These initiatives increased search click-through rates to 94% and lifted the proportion of users finding discovery easy to 73%. ### [Bringing Grab’s Live Activity to Android: Enhancing user experience through custom notifications](https://yomu.fyi/post/bringing-grab-s-live-activity-to-android-enhancing-user-experience-thr.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Jessica Sean - Published: Sep 23, 2024 Grab designed an equivalent to iOS Live Activities for Android to provide real-time order tracking outside the app. Because Android lacks Apple's native ActivityKit push token system, the team substituted push tokens with placeholder values to maintain technical consistency across platforms and preserve backend push targeting via their Hedwig service. For the user interface, engineering selected custom notifications over floating views because custom notifications avoid intrusive screen usage and do not require the 'Draw over other apps' permission. The client implementation separates responsibilities across dedicated classes: LiveActivityIntegrationManager handles token registration across business verticals, LiveActivityAttributes encapsulates UI configuration, and LiveActivityManager maps payloads to Android NotificationManager instances. The solution launched for Food, Mart, Express, and Transport verticals. ### [Unveiling the process: The creation of our powerful campaign builder](https://yomu.fyi/post/unveiling-the-process-the-creation-of-our-powerful-campaign-builder.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Jie Zhang - Published: Sep 10, 2024 Grab details the event processing architecture behind Trident, its internal marketing campaign platform that evaluates If This, Then That (IFTTT) logic over Kafka streams. The core processing unit is a treatment consisting of an event, optional conditions, and actions. Complex campaign capabilities—such as counters, limits, and delays across multiple hours via recursive SQS message scheduling—are assembled from multiple coordinated treatments. To simplify campaign creation, Grab introduced a flowchart-like visual builder represented as a JSON node tree that compiles recursively into treatments while persisting node-to-treatment mappings to reconcile edits over time. ### [Chimera Sandbox: A scalable experimentation and development platform for Notebook services](https://yomu.fyi/post/chimera-sandbox-a-scalable-experimentation-and-development-platform-fo.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Arkar Min Aung - Published: Aug 27, 2024 Grab's Chimera team built Chimera Sandbox to address resource accessibility challenges and enable rapid machine learning experimentation across technical and non-technical teams. The platform runs on multi-cluster Kubernetes infrastructure capable of scaling to thousands of nodes per cluster during peak loads. Users can provision customizable Docker environments with varying CPU and GPU configurations, collaborate in real time, and query data using Spark-on-Kubernetes integrations and helper magic cells. The workspace integrates with Grab's AI Gateway for LLM access alongside tools like MLflow, Optuna, RStudio, and LabelStudio on shared storage, backed by user-level cost attribution. ### [How we improved translation experience with cost efficiency](https://yomu.fyi/post/how-we-improved-translation-experience-with-cost-efficiency.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Jie Zhang - Published: Aug 5, 2024 Grab faced translation quality and cost challenges in its booking chat system when tourism resumed after COVID restrictions eased. Device language settings proved unreliable for determining message language, prompting the team to use Lingua alongside booking context heuristics and third-party fallbacks for real-time detection. To replace expensive and inaccurate general translation APIs, Grab distilled a large open-source Hugging Face model into lightweight, language-specific models trained on ten million synthetic chat examples. A post-translation validation step verifies non-translatable entities such as numbers and emojis before falling back to external services if mutations occur. Caching layers were also introduced across translation paths to curb redundant on-the-fly execution. ### [LLM-powered data classification for data entities at scale](https://yomu.fyi/post/llm-powered-data-classification-for-data-entities-at-scale.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Hualin Liu - Published: Jul 15, 2024 Grab needed to classify sensitive data at the table and column level across petabytes of database tables and streaming schemas. Manual schema-tiering campaigns had resulted in half of all schemas receiving overly strict Tier 1 access controls, while an initial automated service using regex patterns and third-party machine learning produced high false-positive rates and lacked customizability. To address this, the Caspian data engineering and governance teams enhanced their internal orchestration service, Gemini, by integrating GPT-3.5 via Azure OpenAI. Gemini aggregates classification requests into mini-batches, handles API rate limits, and uses prompt engineering—including few-shot examples, curated tag libraries, and explicit JSON DTO schemas—to reliably tag columns for data owner verification. ### [Profile-guided optimisation (PGO) on Grab services](https://yomu.fyi/post/profile-guided-optimisation-pgo-on-grab-services.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Yonghao Hu - Published: Jun 5, 2024 Profile-guided optimisation (PGO) improves Go application performance by feeding production CPU profile data into subsequent compiler builds. Teams evaluated PGO across multiple Grab services, including TalariaDB and an orchestrator within the Catwalk service, using Go 1.20 and Go 1.21 builds. TalariaDB saw substantial resource savings after compiling Docker images with a 360-second pprof profile, resulting in at least a 10% drop in CPU usage, a 30% drop in memory consumption, and a 38% reduction in persistent volume usage for event ingestion queues. Conversely, testing on Catwalk showed only a 5% performance gain after adjusting the profiling duration from 59 seconds to 6 minutes, an outcome deemed insufficient to justify pipeline modifications. Broader adoption across monorepo services remains constrained until the central build process supports PGO arguments and automated profile retrieval. ### [How we evaluated the business impact of marketing campaigns](https://yomu.fyi/post/how-we-evaluated-the-business-impact-of-marketing-campaigns.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Jie Zhang - Published: May 23, 2024 Grab needed a scalable, centralised way to evaluate the business impact of millions of marketing messages sent across expanding campaigns. Manual analysis by marketing analysts produced inconsistent results and failed to scale, leading to the creation of multi-touch attribution and controlled experiment methodologies. The team designed a Marketing attribution model (MAM) that credits touchpoints based on relevance and recency, while using stratified hold-out groups to assess single campaign lift against baseline metrics. Statistical significance calculations, implemented using the jStat library based on the central limit theorem, ensure observed business metric differences like gross merchandise value are not random. The system provides campaign managers with direct access to impact analyses while the team works on challenges surrounding smaller sample sizes and hold-out group constraints. ### [No version left behind: Our epic journey of GitLab upgrades](https://yomu.fyi/post/no-version-left-behind-our-epic-journey-of-gitlab-upgrades.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Saurabh Vajpayee - Published: May 3, 2024 Grab's self-hosted GitLab instance fell approximately 14 months behind official releases, serving thousands of engineers and monorepos reaching up to 39TB in total footprint. To address security updates and system stability, the team established a structured upgrade routine using Terraform, Packer, and Ansible across GitLab's 5,000-user reference architecture. They implemented staged deployments, upgraded stateful Gitaly nodes via in-place rotation, and resolved primary node clustering bottlenecks under Praefect. By utilizing official GitLab Upgrade Paths and parallelizing component deployments, the team caught up on 24 months of releases in 11 months and reduced release lag from 396 days down to 35 days. ### [Ensuring data reliability and observability in risk systems](https://yomu.fyi/post/ensuring-data-reliability-and-observability-in-risk-systems.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Yi Ni Ong - Published: Apr 23, 2024 Grab's in-house risk management platform, GrabDefence, relies on ingesting large volumes of upstream data to drive real-time heuristic rules and data science models. To prevent data discrepancies and missing information from degrading fraud detection, the team implemented a real-time data observability pipeline. The solution uses Apache Flink SQL alongside custom table functions like JSONEXPLOAD to flatten nested JSON structures and aggregate metrics over 5-minute tumbling windows. Processed metrics are sent to Datadog, where counters are grouped by source stream and evaluated using anomaly detection algorithms. Alerts are routed directly to Slack, reducing anomaly detection latency from days or weeks down to within the same day or hour. ### [Grab Experiment Decision Engine - a Unified Toolkit for Experimentation](https://yomu.fyi/post/grab-experiment-decision-engine-a-unified-toolkit-for-experimentation.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Ruike Zhang - Published: Apr 9, 2024 Grab's Experimentation platform tests thousands of experimental variants weekly, but analyses were largely handled manually using disparate tools unsuited to multi-sided marketplace nuances. To standardize workflows and eliminate reliability issues, Grab developed the GrabX Decision Engine, an internally open-sourced Python toolkit integrated with its Automated Experiment Analysis application. The framework includes a Trusted Advisor component that automates power analysis and sample size estimation while adjusting for clustered standard errors across metric data types. It also bundles a post-experiment statistical toolbox executing default and custom evaluations, ranging from non-parametric tests to regression models that control for confounders and fixed effects. This unified platform institutionalizes experimentation best practices across teams, streamlining collaboration and ensuring consistent statistical inference. ### [Iris - Turning observations into actionable insights for enhanced decision making](https://yomu.fyi/post/iris-turning-observations-into-actionable-insights-for-enhanced-decisi.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Huong Vuong - Published: Apr 3, 2024 Standard infrastructure monitoring tools track CPU and memory usage at the host or container level, obscuring the precise resource consumption of individual Spark jobs on shared instances. To achieve granular observability, the Iris platform gathers job metrics directly at the Java Virtual Machine level without requiring changes to user application code. Data collection relies on Uber's JVM Profiler for deep JVM, thread, and memory pool diagnostics alongside sparkMeasure for stage and task execution statistics. Collected metrics route asynchronously through an Apache Kafka queue to avoid execution bottlenecks, feeding a Telegraf, InfluxDB, and Grafana stack for real-time monitoring while archiving to a data lake for offline analytics. This multi-tiered telemetry enables engineering teams to identify over- or under-provisioned jobs and optimize ETL resource allocations. ### [Android App Size at Scale with Project Bonsai](https://yomu.fyi/post/android-app-size-at-scale-with-project-bonsai.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Nguyen Van Minh - Published: Mar 1, 2024 Grab launched Project Bonsai to optimize the size of its Android superapp, which encompassed over four million lines of code across 1,500 modules. Because large app sizes negatively impact user acquisition on low-end devices with constrained bandwidth, the engineering team structured its optimization strategy around measurement, reduction, and containment. The team developed App Sizer, a custom CI-integrated tool that tracks binary compositions and module size contributions in Grafana. Initial reductions targeted Java and Kotlin dex files, resolving bloated transitive R classes by upgrading the Android Gradle Plugin to eliminate overly broad R8 retention rules. These initiatives achieved a 26% reduction in app download size while simultaneously decreasing overall disk footprint. ### [Enabling near real-time data analytics on the data lake](https://yomu.fyi/post/enabling-near-real-time-data-analytics-on-the-data-lake.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Shi Kai Ng - Published: Feb 23, 2024 Traditional data lake setups using Parquet on Hive metastores struggle with frequent updates and long pipeline intervals, introducing significant latency for ad hoc queries. Grab solved this by implementing Apache Hudi to support near real-time analytics across bounded relational databases and unbounded Kafka streams. For high-throughput sources, Flink streams Avro log files to Merge On Read tables and generates compaction plans for asynchronous Spark writers. Low-throughput workloads leverage Copy On Write tables, while relational database sources ingest binlogs via Flink Change Data Capture connectors. This architecture reduced analytics data latency to the minute level without overloading production databases and Kafka clusters. ### [The journey of building a comprehensive attribution platform](https://yomu.fyi/post/the-journey-of-building-a-comprehensive-attribution-platform.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Kang Huang - Published: Feb 20, 2024 Grab needed to modernize its marketing analytics from manual ad hoc queries and high data latency to a platform supporting real-time attribution for pricing models like cost per order. The engineering team initially deployed a pure stream-processing engine using Kappa architecture, Kafka, ScyllaDB, and Redis, which reduced latency from days to minutes and merged ads and promo touchpoints. However, stream-only processing faced high costs, out-of-order event issues, and difficulties running multi-touch models across longer historical windows. Grab transitioned to a Lambda architecture pairing Coban stream processing with Spark-based batch ETL and Amazon S3. This hybrid design separated real-time operational metrics from historical batch reporting, cutting real-time processing costs by approximately 25% while maintaining under 1% data discrepancy. ### [Managing dynamic marketplace content at scale: Grab's approach to content moderation](https://yomu.fyi/post/managing-dynamic-marketplace-content-at-scale-grab-s-approach-to-conte.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Poonam Gambhire - Published: Feb 1, 2024 Grab manages content moderation across GrabFood and GrabMart, where merchants update over 100,000 item listings daily across eight countries. The platform must adhere to internal rules, local government regulations, and external platform guidelines across these markets. To handle scale and regional nuance, Grab uses an in-house automated system powered by algorithms and machine learning to scan daily listings for violations. Items flagged as requiring subjective judgment or cultural awareness are routed to human moderators for manual review. Additionally, Grab collaborates with Google to maintain compliance with Play Store policies. ### [Rethinking Stream Processing: Data Exploration](https://yomu.fyi/post/rethinking-stream-processing-data-exploration.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Shi Kai Ng - Published: Jan 31, 2024 Online data materialised in Kafka topics at Grab traditionally remained uninspected until ingested into the Data Lake as offline data, introducing latency and making stream processing applications difficult to develop. To enable earlier data exploration, Grab adopted Apache Zeppelin notebooks integrated with an internal data streaming platform. Zeppelin uses a Flink interpreter to convert interactive SQL queries into Flink jobs submitted to a Flink session cluster. The platform dynamically translates Protobuf schemas into SQL Data Definition Language statements to simplify table discovery. Security and compliance risks are addressed by combining mutual TLS authentication with Strimzi's Open Policy Agent to enforce read-only access policies and audit logging on Kafka topics. ### [Kafka on Kubernetes: Reloaded for fault tolerance](https://yomu.fyi/post/kafka-on-kubernetes-reloaded-for-fault-tolerance.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Fabrice Harbulot - Published: Dec 26, 2023 Grab's real-time data streaming platform, Coban, operates Kafka on AWS Elastic Kubernetes Service using Strimzi, allocating an entire EC2 worker node with NVMe instance store volumes to each broker. An initial architecture suffered from client connection errors, broken Network Load Balancer target groups, and zombie Persistent Volume Claims when worker nodes terminated. To achieve automated fault tolerance, the team integrated the AWS Node Termination Handler in Queue Processor mode with Auto Scaling lifecycle hooks, ensuring Kafka receives a SIGTERM to migrate partition leadership gracefully before shutdown. They also introduced the Kubernetes Cluster Autoscaler to dynamically provision replacement nodes during maintenance events and used the AWS Load Balancer Controller with TargetGroupBinding custom resources to dynamically update load balancer targets using IP mode. ### [Championing CyberSecurity: Grab's bug bounty programme in 2023](https://yomu.fyi/post/championing-cybersecurity-grab-s-bug-bounty-programme-in-2023.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Nathaniel Callens - Published: Dec 19, 2023 Grab reviewed the performance and milestones of its security bug bounty programme for 2023, marking eight years of running the initiative. Operating quarterly campaigns through HackerOne alongside dedicated internal teams, the company processed approximately 1,000 vulnerability submissions over the year, including 400 reports during Q1 alone. Cumulative payouts to external security researchers surpassed $700,000 since the programme's 2015 launch. To adapt to an expanding service portfolio, Grab broadened its scope to encompass joint ventures and acquisitions while introducing diversified financial and recognition-based rewards. ### [Sliding window rate limits in distributed systems](https://yomu.fyi/post/sliding-window-rate-limits-in-distributed-systems.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Naveen Kumar Jakuva Premkumar - Published: Dec 14, 2023 Marketing communications across Grab's user base risked causing notification overload and consent revocations. To enforce personalized daily and weekly frequency caps across more than 270 million users, the team addressed segment membership storage and communication rate limiting. They adopted roaring bitmaps instead of Bloom filters to compactly store user segment data without hash collisions or costly rebuilds upon deletion. For frequency capping, they chose Amazon ElastiCache for Redis over DynamoDB, executing a sliding log rate limiting algorithm directly on the cluster using Lua scripts and sorted sets. Timestamps are stored as sorted set scores, and historical data is cleaned up via eviction ranges to prevent unbounded memory growth. [Newer posts](https://yomu.fyi/company/grab/page/3.md) · [Older posts](https://yomu.fyi/company/grab/page/5.md)