Loading post…
GrabArkar Min AungAnnouncement
Chimera Sandbox: A scalable experimentation and development platform for Notebook services
Summary
Grab's Chimera team built Chimera Sandbox to address resource accessibility challenges and enable rapid machine learning experimentation across technical and non-technical teams. The platform runs on multi-cluster Kubernetes infrastructure capable of scaling to thousands of nodes per cluster during peak loads. Users can provision customizable Docker environments with varying CPU and GPU configurations, collaborate in real time, and query data using Spark-on-Kubernetes integrations and helper magic cells. The workspace integrates with Grab's AI Gateway for LLM access alongside tools like MLflow, Optuna, RStudio, and LabelStudio on shared storage, backed by user-level cost attribution.
Context
Rapid machine learning and Generative AI experimentation across Grab was hindered by limited accessibility to compute resources like GPU instances and specialized services for building LLM-powered applications.
Approach / What changed
Grab built Chimera Sandbox, a multi-tenant workspace platform on Kubernetes clusters supporting custom Docker images, integrated Spark engines, an AI Gallery of prebuilt notebooks, and unified access to internal ML tools and LLM gateways.
Takeaways
The compute infrastructure runs across multiple Kubernetes clusters that can scale up to thousands of nodes per cluster during peak usage.
Integrated Spark tooling includes a SparkConnect Kernel, a custom Spark-on-Kubernetes operator, and a %%spark_sql magic cell for direct querying without manual session initialization.
Each multi-tenant workspace includes user-level cost showback and chargeback attribution to track expenditure on GPU and compute instances.
Source and freshness
- Source
- engineering.grab.com
- Published
- Added to Yomu
- Last updated
Summaries are generated from the original by a language model. Check the original before relying on it.
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
The best engineering reads, once a week
One useful email. Unsubscribe anytime.