Evolution of Catwalk: Model serving platform at Grab

Summary

Grab developed and scaled Catwalk, an internal machine learning model serving platform, to address operational bottlenecks, low resource utilization, and deployment friction between data scientists and backend engineers. The platform transitioned from an admin-managed TensorFlow Serving setup into a low-code self-service system supporting PyTorch and ONNX, before replacing complex Helm charts with Kubernetes Custom Resource Definitions for declarative, blue-green deployment orchestration. To support complex business workflows and multi-model applications, Grab subsequently introduced Catwalk Orchestrator with bundled deployments that allow individual services to scale independently. Across two years, the orchestrator architecture expanded to 200 deployed applications serving approximately 1,400 production machine learning models.

Takeaways

1. 1

   Replacing Helm charts with Kubernetes Custom Resource Definitions enabled automated blue-green deployments, capacity management, and constraint validation through a custom state machine.

2. 2

   Using bundled deployments allowed each model and orchestration component in an ML application to operate as a separate Kubernetes deployment with independent scaling.

3. 3

   Adopting a high-code orchestrator alongside mandatory staging load testing expanded platform scale to 200 orchestrators serving 1,400 models.