Catwalk: Serving Machine Learning Models at Scale

Summary

As machine learning adoption expanded at Grab, individual teams created fragmented model serving solutions that duplicated engineering effort and required data scientists to handle underlying infrastructure. To resolve these inefficiencies, Grab developed Catwalk, a self-service machine learning model serving platform. The system runs TensorFlow Serving containers across a managed Kubernetes cluster integrated with Grab's observability stack. Data scientists deploy or update models simply by saving files using the tf.saved_model API to dedicated Amazon S3 buckets, while Kubernetes automates orchestration, ingress routing, and pod autoscaling. Catwalk abstracts server management away from data scientists, shortens deployment timelines, and provides high availability during model version rollouts.

Context

Teams at Grab were independently building custom model serving solutions, which created duplicated engineering effort and required data scientists to understand complex infrastructure instead of focusing on machine learning.

Approach / What changed

Grab built Catwalk, a self-serve platform running TensorFlow Serving in containers on a managed Kubernetes cluster, allowing data scientists to deploy and update models simply by uploading exported model files to Amazon S3 buckets.

Takeaways

• TensorFlow Serving continuously monitors the configured Amazon S3 model bucket to automatically load newly uploaded model versions without downtime.
• Kubernetes manages model workload isolation and auto-scaling by assigning TensorFlow Serving pods to cluster nodes with sufficient CPU and memory resources.
• Catwalk provisions Kubernetes Deployment, Service, and Ingress resources alongside an ingress controller to configure load balancing and expose HTTP endpoints for each model.

Related reading

• Grab · Oct 1, 2024

  Evolution of Catwalk: Model serving platform at Grab

  Grab developed and scaled Catwalk, an internal machine learning model serving platform, to address operational bottlenecks, low resource utilization, and deployment friction between data scientists and backend engineers. The platform transitioned from an admin-managed TensorFlow Serving setup into a low-code self-service system supporting PyTorch and ONNX, before replacing complex Helm charts with Kubernetes Custom Resource Definitions for declarative, blue-green deployment orchestration. To support complex business workflows and multi-model applications, Grab subsequently introduced Catwalk Orchestrator with bundled deployments that allow individual services to scale independently. Across two years, the orchestrator architecture expanded to 200 deployed applications serving approximately 1,400 production machine learning models.

  Vishal Sharma

• Grab · Oct 21, 2025

  Modernising Grab’s model serving platform with NVIDIA Triton Inference Server

  Grab's machine learning serving platform, Catwalk, experienced mounting technical debt, elevated latency, and rising costs from maintaining disparate inference engines for various frameworks. To address these limitations, the team adopted NVIDIA Triton Inference Server to establish a unified engine, starting with an ONNX migration. They built a custom Triton manager component featuring a proxy layer to translate legacy API requests alongside a server manager that handles model downloads, verification, configuration, and health checks. Within ten days, over half of online deployments migrated seamlessly without requiring client code modifications. The transition reduced p90 latency from 120ms to 20ms on large transformer models and cut average infrastructure spend by approximately 20 percent across evaluated services.

  Daniel Tai

• Grab · Aug 1, 2026

  How AI is transforming analytics at Grab

  Grab is restructuring its analytics operations using a five-level AI autonomy ladder, transitioning analysts from manual artifact creation to problem framing and decision governance. The architecture leverages domain-specific systems like Spartan to process natural language queries through certified metric indexes and Scarlet to triage and repair failing data pipelines. To prevent agent hallucinations, ContextIQ manages context lifecycles by automatically updating metric definitions, SQL references, and golden-dataset test cases when instrumentation changes or failures occur. Furthermore, data teams use an internal portal called BriX to configure custom analytics surfaces and automated root-cause analysis commentaries using reusable Model Context Protocol connections. Autonomy scales mechanical query and validation tasks while keeping human oversight focused on canonical metric definitions and strategic sign-offs.

  Maanas Prabhakar

• Grab · Sep 26, 2025

  User foundation models for Grab

  Traditional recommendation systems at Grab relied on siloed, manually engineered features that struggled to capture sequential interaction data and cross-service user intent across its superapp ecosystem. To resolve these limitations, Grab developed a custom user foundation model built on a transformer architecture designed to learn from both tabular profile data and time-series clickstream interactions. The architecture unifies diverse data modalities—including categorical identifiers, numerical values, text, and geographical coordinates—by framing inputs as key-value token pairs. Custom positional embeddings and attention masks ensure the model processes tabular attributes as unordered sets while maintaining the sequential order of time-series events. The pre-trained model generates holistic user embeddings and supports direct fine-tuning across downstream tasks such as fraud detection, churn prediction, and advertisement optimization.

  Abhinav Rai