Runtime Roundup: VM Sandboxes, Multi-node clusters, and more

Summary

Modal announced several new features at its inaugural Runtime conference. VM Sandboxes provide agents with full Linux computer access, supporting Docker stacks, databases, graphical environments, and kernel experimentation, already used at Linear, Legora, and Snorkel for coding agents and evals. Sandbox Sidecars run alongside main sandboxes with isolation but share a host, allowing agent code in the main sandbox while trusted operations run in a sidecar the agent cannot access, avoiding network round trips in execution-heavy loops. Modal Clusters, now generally available via @modal.clustered, enable multi-node jobs for training or serving trillion-parameter models, powering post-training at Decagon and real-time inference at Runway. Sticky Sessions maintain client state by routing to the same container using a session token, critical for voice model serving, while Endpoint Candidates allow mirroring production traffic to compare quality, latency, throughput, and cost before promoting.

Takeaways

1. 1

   VM Sandboxes give agents a full Linux computer environment, supporting Docker stacks, databases, dev servers, graphical environments, and kernel work, already used by Linear, Legora, and Snorkel for coding agents and evals.

2. 2

   Sandbox Sidecars run alongside the main sandbox with isolation but share a host, letting agent code run while credentials and harness logic run in an inaccessible sidecar, cutting network round trips.

3. 3

   Modal Clusters via @modal.clustered enable multi-node coordinated containers for training or serving trillion-parameter models, used by Decagon for post-training and Runway for real-time inference.

FAQ