# How we evaluated the business impact of marketing campaigns

[Grab](https://yomu.fyi/company/grab) · Jie Zhang · May 23, 2024

**Type:** Problem & solution

## Summary

Grab needed a scalable, centralised way to evaluate the business impact of millions of marketing messages sent across expanding campaigns. Manual analysis by marketing analysts produced inconsistent results and failed to scale, leading to the creation of multi-touch attribution and controlled experiment methodologies. The team designed a Marketing attribution model (MAM) that credits touchpoints based on relevance and recency, while using stratified hold-out groups to assess single campaign lift against baseline metrics. Statistical significance calculations, implemented using the jStat library based on the central limit theorem, ensure observed business metric differences like gross merchandise value are not random. The system provides campaign managers with direct access to impact analyses while the team works on challenges surrounding smaller sample sizes and hold-out group constraints.

## Context

Manual marketing impact analysis did not scale with an increasing number of marketing campaigns at Grab, and different analyst methodologies caused inconsistent assessments over time.

## Approach / What changed

Grab introduced a centralised solution combining a multi-touch Marketing Attribution Model weighting relevance and recency with stratified hold-out testing and statistical significance calculations using jStat.

## Takeaways

- Grab's Marketing Attribution Model assigns multi-touch conversion credit across channels by weighting the relevance and recency of each marketing message.
- Evaluating single-campaign lift uses stratified hold-out (control) and treatment groups to aggregate metrics such as gross merchandise value divided by user count.
- Statistical significance is derived from the central limit theorem and calculated programmatically using the jStat library to filter out random variations.

**Tags:** [Data Pipelines](https://yomu.fyi/topic/data-pipelines), [Scalability](https://yomu.fyi/topic/scalability), [Testing](https://yomu.fyi/topic/testing)

[Read original post](https://engineering.grab.com/evaluate-business-impact-of-marketing-campaigns)