# Chimera Sandbox: A scalable experimentation and development platform for Notebook services

[Grab](https://yomu.fyi/company/grab) · Arkar Min Aung · Aug 27, 2024

**Type:** Announcement

## Summary

Grab's Chimera team built Chimera Sandbox to address resource accessibility challenges and enable rapid machine learning experimentation across technical and non-technical teams. The platform runs on multi-cluster Kubernetes infrastructure capable of scaling to thousands of nodes per cluster during peak loads. Users can provision customizable Docker environments with varying CPU and GPU configurations, collaborate in real time, and query data using Spark-on-Kubernetes integrations and helper magic cells. The workspace integrates with Grab's AI Gateway for LLM access alongside tools like MLflow, Optuna, RStudio, and LabelStudio on shared storage, backed by user-level cost attribution.

## Context

Rapid machine learning and Generative AI experimentation across Grab was hindered by limited accessibility to compute resources like GPU instances and specialized services for building LLM-powered applications.

## Approach / What changed

Grab built Chimera Sandbox, a multi-tenant workspace platform on Kubernetes clusters supporting custom Docker images, integrated Spark engines, an AI Gallery of prebuilt notebooks, and unified access to internal ML tools and LLM gateways.

## Takeaways

- The compute infrastructure runs across multiple Kubernetes clusters that can scale up to thousands of nodes per cluster during peak usage.
- Integrated Spark tooling includes a SparkConnect Kernel, a custom Spark-on-Kubernetes operator, and a %%spark\_sql magic cell for direct querying without manual session initialization.
- Each multi-tenant workspace includes user-level cost showback and chargeback attribution to track expenditure on GPU and compute instances.

**Tags:** [Developer Experience](https://yomu.fyi/topic/developer-experience), [Kubernetes](https://yomu.fyi/topic/kubernetes), [LLMs](https://yomu.fyi/topic/llm), [Machine Learning](https://yomu.fyi/topic/machine-learning), [Scalability](https://yomu.fyi/topic/scalability)

[Read original post](https://engineering.grab.com/chimera-sandbox)