# Evolution of Catwalk: Model serving platform at Grab

[Grab](https://yomu.fyi/company/grab) · Vishal Sharma · Oct 1, 2024

**Type:** Problem & solution

## Summary

Grab developed and scaled Catwalk, an internal machine learning model serving platform, to address operational bottlenecks, low resource utilization, and deployment friction between data scientists and backend engineers. The platform transitioned from an admin-managed TensorFlow Serving setup into a low-code self-service system supporting PyTorch and ONNX, before replacing complex Helm charts with Kubernetes Custom Resource Definitions for declarative, blue-green deployment orchestration. To support complex business workflows and multi-model applications, Grab subsequently introduced Catwalk Orchestrator with bundled deployments that allow individual services to scale independently. Across two years, the orchestrator architecture expanded to 200 deployed applications serving approximately 1,400 production machine learning models.

## Context

Prior ad-hoc ML model serving at Grab caused operational overhead, low resource utilization of around 1%, duplicated engineering efforts, and friction between data scientists and backend teams.

## Approach / What changed

Grab evolved Catwalk from a managed TensorFlow Serving platform into a self-service system, replaced Helm charts with Kubernetes Custom Resource Definitions for safer deployments, and built Catwalk Orchestrator with bundled Kubernetes deployments.

## Takeaways

- Replacing Helm charts with Kubernetes Custom Resource Definitions enabled automated blue-green deployments, capacity management, and constraint validation through a custom state machine.
- Using bundled deployments allowed each model and orchestration component in an ML application to operate as a separate Kubernetes deployment with independent scaling.
- Adopting a high-code orchestrator alongside mandatory staging load testing expanded platform scale to 200 orchestrators serving 1,400 models.

**Tags:** [Architecture](https://yomu.fyi/topic/architecture), [Deployment](https://yomu.fyi/topic/deployment), [Kubernetes](https://yomu.fyi/topic/kubernetes), [Machine Learning](https://yomu.fyi/topic/machine-learning), [Scalability](https://yomu.fyi/topic/scalability)

[Read original post](https://engineering.grab.com/catwalk-evolution)