Loading post…
ModalProblem & solution
Unpacking sandbox startup latency: why started ≠ ready
Summary
Sandbox startup benchmarks measure container boot time, but in production application-level initialization after boot dominates, making container boot the smallest variable. Modal defines a lifecycle with events Created, Scheduled, Started, and Ready, where Ready indicates completion of tasks like git pulls or dependency installs. The gap between Started and Ready can be multiple seconds, directly affecting user-perceived latency for coding agents needing cloned repos and running services. To reduce perceived latency, Modal recommends warm pools that pre-initialize sandboxes using a Queue, combined with Directory Snapshots for per-project customization without rebuilding the environment. Readiness Probes, now generally available, allow defining a shell command that exits 0 or a TCP port that accepts connections, with wait_until_ready() blocking until the sandbox is truly ready, simplifying warm pool producer code and providing dashboard events for observability.
Context
The post explains that the gap between container started and application ready is often the largest part of startup latency, and that benchmarks ignore it. It provides examples from coding agents, vibe coding platforms, and RL training, and introduces Modal’s tools to address the issue.
Approach / What changed
Modal’s approach includes warm pools using a Queue to pre-initialize sandboxes, Directory Snapshots for per-project customization without rebuilding the environment, and Readiness Probes (shell command or TCP port check) with wait_until_ready() and dashboard events for observability.
Takeaways
Container boot time is the smallest variable in sandbox startup; application-level setup like git pulls and dependency installs dominates perceived latency.
Warm pools pre-initialize sandboxes in the background, and Directory Snapshots allow per-project customization without rebuilding the environment.
Readiness Probes provide a first-class way to verify when a sandbox is truly ready, using shell commands or TCP port checks, with dashboard events for observability.
FAQ
What is the difference between Started and Ready in sandbox startup?
Started means the container is live and you can exec commands, but Ready means application-level initialization (e.g., git pull, dependency install, server listening) has finished. The gap between them is often multiple seconds and is the dominant part of startup latency.
How do warm pools reduce perceived latency?
Warm pools pre-initialize sandboxes in the background before requests arrive. When a request comes, a pre-warmed sandbox is handed out instantly, so the user doesn't wait for setup. Directory Snapshots allow per-project customization without rebuilding the environment.
What are Readiness Probes and how do they work?
Readiness Probes are a first-class way to check if a sandbox is truly ready. You define a shell command that should exit 0 or a TCP port that should accept connections, and Modal polls until the check passes. wait_until_ready() blocks until the sandbox is ready, simplifying warm pool producer code.
Source and freshness
- Source
- modal.com
- Published
- Added to Yomu
Summaries are generated from the original by a language model. Check the original before relying on it.
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
The best engineering reads, once a week
One useful email. Unsubscribe anytime.