# Automating Multi-Armed Bandit testing during feature rollout

[Grab](https://yomu.fyi/company/grab) · Weicheng Zhu · Sep 1, 2021

**Type:** Problem & solution

## Summary

Traditional feature rollouts and Multi-Armed Bandit testing operate as separate workflows that often depend on delayed offline analysis. To eliminate manual intervention, the Multi-Armed Bandit Optimiser automates testing concurrently during feature rollouts by responding to minute-level feedback metrics. The architecture connects Kafka Streams data processing, a metrics server with Spark jobs, and an adaptive rollout module updating online experimentation configurations. Candidate models are evaluated via Thompson Sampling on Beta distributions, with Monte Carlo simulations determining traffic allocation across user entities. In production for the GrabFood recommendation widget, the system optimizes the Effective Conversion Rate over a 30-minute window and includes fallback distribution logic.

## Context

Traditional software development handles Multi-Armed Bandit testing and new feature rollouts as separate processes that rely on offline experiment analysis at T+N, requiring manual intervention from data scientists and product managers to update model choices.

## Approach / What changed

The Multi-Armed Bandit Optimiser automates bandit testing simultaneously during rollouts using a Kafka Streams pre-processing framework, a metrics feedback loop powered by Spark and Stalker, Thompson Sampling over Beta distributions, and Monte Carlo simulations.

## Takeaways

- The Multi-Armed Bandit Optimiser evaluates arms using Batched Thompson Sampling over Beta distributions combined with Monte Carlo simulations utilizing either a Max or Mean strategy.
- For GrabFood food recommendation models, optimization targets the Effective Conversion Rate—calculated as widget checkouts divided by views multiplied by coverage rate—aggregated over 30-minute windows.
- The rollout policy routes traffic by user entity ID and automatically reverts to a default 50-50% distribution between models if scheduled optimization jobs fail.

**Tags:** [Architecture](https://yomu.fyi/topic/architecture), [Data Pipelines](https://yomu.fyi/topic/data-pipelines), [Kafka](https://yomu.fyi/topic/kafka), [Recommendation Systems](https://yomu.fyi/topic/recommendation-systems), [Streaming](https://yomu.fyi/topic/streaming)

[Read original post](https://engineering.grab.com/multi-armed-bandit-system-recommendation)