DispatchGym: Grab’s reinforcement learning research framework

Summary

DispatchGym is a research framework for reinforcement learning (RL) studies in dispatch systems that match bookings with drivers. It combines an RL algorithm, a dispatch-process simulation, and a Gymnasium API interface, allowing data scientists with basic RL and Python skills to modify and test ideas independently. The simulated environment models booking creation, driver dispatch and movement, and booking completion, prioritizing directional accuracy over exact metric matching; its modular Python architecture uses Numba to reduce runtime overhead. A command-line upload launches the modified code on internal infrastructure as a Spark job, avoiding infrastructure management while preserving flexibility across simulation and optimization components. The framework supported experiments with contextual bandits, including linear, neural-linear, and Gaussian-process variants alongside several action-sampling strategies, to identify combinations of levers and reward functions for dispatch hyperparameter tuning and improved fulfilment efficiency and reliability.

Takeaways

1. 1

   The simulation prioritizes directional accuracy: metric trends should match real-world behavior even when simulated and real metric values differ.

2. 2

   Data scientists can alter both simulation and optimization components locally, then submit the updated code through one CLI command that starts a Spark job.

3. 3

   DispatchGym supported testing linear, neural-linear, and Gaussian-process contextual bandits with epsilon-greedy, Thompson sampling, SquareCB, and FastCB action sampling.