How we evaluated the business impact of marketing campaigns

Summary

Grab needed a scalable, centralised way to evaluate the business impact of millions of marketing messages sent across expanding campaigns. Manual analysis by marketing analysts produced inconsistent results and failed to scale, leading to the creation of multi-touch attribution and controlled experiment methodologies. The team designed a Marketing attribution model (MAM) that credits touchpoints based on relevance and recency, while using stratified hold-out groups to assess single campaign lift against baseline metrics. Statistical significance calculations, implemented using the jStat library based on the central limit theorem, ensure observed business metric differences like gross merchandise value are not random. The system provides campaign managers with direct access to impact analyses while the team works on challenges surrounding smaller sample sizes and hold-out group constraints.

Takeaways

1. 1

   Grab's Marketing Attribution Model assigns multi-touch conversion credit across channels by weighting the relevance and recency of each marketing message.

2. 2

   Evaluating single-campaign lift uses stratified hold-out (control) and treatment groups to aggregate metrics such as gross merchandise value divided by user count.

3. 3

   Statistical significance is derived from the central limit theorem and calculated programmatically using the jStat library to filter out random variations.