Automating Multi-Armed Bandit testing during feature rollout

Summary

Traditional feature rollouts and Multi-Armed Bandit testing operate as separate workflows that often depend on delayed offline analysis. To eliminate manual intervention, the Multi-Armed Bandit Optimiser automates testing concurrently during feature rollouts by responding to minute-level feedback metrics. The architecture connects Kafka Streams data processing, a metrics server with Spark jobs, and an adaptive rollout module updating online experimentation configurations. Candidate models are evaluated via Thompson Sampling on Beta distributions, with Monte Carlo simulations determining traffic allocation across user entities. In production for the GrabFood recommendation widget, the system optimizes the Effective Conversion Rate over a 30-minute window and includes fallback distribution logic.

Takeaways

1. 1

   The Multi-Armed Bandit Optimiser evaluates arms using Batched Thompson Sampling over Beta distributions combined with Monte Carlo simulations utilizing either a Max or Mean strategy.

2. 2

   For GrabFood food recommendation models, optimization targets the Effective Conversion Rate—calculated as widget checkouts divided by views multiplied by coverage rate—aggregated over 30-minute windows.

3. 3

   The rollout policy routes traffic by user entity ID and automatically reverts to a default 50-50% distribution between models if scheduled optimization jobs fail.