The journey of building a comprehensive attribution platform

Summary

Grab needed to modernize its marketing analytics from manual ad hoc queries and high data latency to a platform supporting real-time attribution for pricing models like cost per order. The engineering team initially deployed a pure stream-processing engine using Kappa architecture, Kafka, ScyllaDB, and Redis, which reduced latency from days to minutes and merged ads and promo touchpoints. However, stream-only processing faced high costs, out-of-order event issues, and difficulties running multi-touch models across longer historical windows. Grab transitioned to a Lambda architecture pairing Coban stream processing with Spark-based batch ETL and Amazon S3. This hybrid design separated real-time operational metrics from historical batch reporting, cutting real-time processing costs by approximately 25% while maintaining under 1% data discrepancy.

Takeaways

1. 1

   Kappa architecture reduced attribution data latency from 2–3 days down to a few minutes to support real-time decision-making.

2. 2

   A pure streaming approach struggled with out-of-order events, multi-touch attribution complexity, and scaling costs over longer historical attribution windows.

3. 3

   Transitioning to a Lambda architecture reduced real-time data processing costs by approximately 25% and brought data discrepancies below 1%.