The journey of building a comprehensive attribution platform

Summary

Grab needed to modernize its marketing analytics from manual ad hoc queries and high data latency to a platform supporting real-time attribution for pricing models like cost per order. The engineering team initially deployed a pure stream-processing engine using Kappa architecture, Kafka, ScyllaDB, and Redis, which reduced latency from days to minutes and merged ads and promo touchpoints. However, stream-only processing faced high costs, out-of-order event issues, and difficulties running multi-touch models across longer historical windows. Grab transitioned to a Lambda architecture pairing Coban stream processing with Spark-based batch ETL and Amazon S3. This hybrid design separated real-time operational metrics from historical batch reporting, cutting real-time processing costs by approximately 25% while maintaining under 1% data discrepancy.

Context

Grab relied on ad hoc queries and manual data preparation that suffered from 2–3 day latencies, discrepant results, and recurring pipeline issues while needing near real-time attribution for cost per order pricing.

Approach / What changed

Grab built a streaming Kappa attribution engine with Kafka and Druid, then evolved it into a Lambda architecture that combined an in-house Coban real-time streaming layer with Spark batch ETL jobs and Amazon S3 storage.

Takeaways

• Kappa architecture reduced attribution data latency from 2–3 days down to a few minutes to support real-time decision-making.
• A pure streaming approach struggled with out-of-order events, multi-touch attribution complexity, and scaling costs over longer historical attribution windows.
• Transitioning to a Lambda architecture reduced real-time data processing costs by approximately 25% and brought data discrepancies below 1%.

Related reading

• Grab · Sep 22, 2023

  Stepping up marketing for advertisers: Scalable lookalike audience

  Grab's legacy lookalike audience generation platform suffered from long creation SLAs of two working days, high costs, and low weekly update frequencies. To resolve these bottlenecks, the engineering team designed an embedding-based platform powered by an in-memory retrieval service and automated update pipelines. The system creates audience representations by averaging constituent passenger embeddings, determining user membership through real-time cosine score thresholds. To eliminate feature store latency while fitting all embeddings into memory, a hash-based compression method cuts passenger embedding storage needs by roughly 90%. Consequently, audience availability dropped to within 15 minutes of campaign creation, audience generation costs fell by 98%, and ad impressions and clicks doubled.

  William Wu

• Grab · Feb 1, 2024

  Managing dynamic marketplace content at scale: Grab's approach to content moderation

  Grab manages content moderation across GrabFood and GrabMart, where merchants update over 100,000 item listings daily across eight countries. The platform must adhere to internal rules, local government regulations, and external platform guidelines across these markets. To handle scale and regional nuance, Grab uses an in-house automated system powered by algorithms and machine learning to scan daily listings for violations. Items flagged as requiring subjective judgment or cultural awareness are routed to human moderators for manual review. Additionally, Grab collaborates with Google to maintain compliance with Play Store policies.

  Poonam Gambhire

• Grab · Sep 23, 2024

  Bringing Grab’s Live Activity to Android: Enhancing user experience through custom notifications

  Grab designed an equivalent to iOS Live Activities for Android to provide real-time order tracking outside the app. Because Android lacks Apple's native ActivityKit push token system, the team substituted push tokens with placeholder values to maintain technical consistency across platforms and preserve backend push targeting via their Hedwig service. For the user interface, engineering selected custom notifications over floating views because custom notifications avoid intrusive screen usage and do not require the 'Draw over other apps' permission. The client implementation separates responsibilities across dedicated classes: LiveActivityIntegrationManager handles token registration across business verticals, LiveActivityAttributes encapsulates UI configuration, and LiveActivityManager maps payloads to Android NotificationManager instances. The solution launched for Food, Mart, Express, and Transport verticals.

  Jessica Sean

• Grab · Mar 31, 2023

  Evolution of quality at Grab

  As Grab expanded its superapp, manual QA sign-offs led to prolonged testing cycles and a peak in major production defects around 2019. To counter defect leakage and improve delivery pace, the engineering organization transitioned to a shift-left testing strategy across its software development lifecycle. Quality engineers began writing Given/When/Then acceptance tests prior to coding so that software engineers could execute them during development, accompanied by Definition of Ready and Definition of Done standards. The team also structured a multi-layer test strategy spanning unit tests, pipeline-executed UI component tests with mocked APIs, backend integration tests, and a custom internal end-to-end framework. Between 2019 and 2022, these changes reduced major and critical production issues by 60% and development-phase critical bugs by 40%.

  Abby Alcantara