Loading post…
ModalPeyton WaltersAnnouncement
Modal Clusters are generally available
Summary
Modal announced general availability of Modal Clusters, a serverless multi-node compute primitive for AI workloads that provides up to 6.4 Tbps InfiniBand RDMA per node with PyTorch and NCCL support. A new gang scheduler places N nodes simultaneously by observing the fleet, assigning all pending clusters, and provisioning capacity when needed. RDMA support is fully automated behind a single rdma=True flag, with Modal contributing RDMA support to the gVisor container runtime and upstreaming the changes. Customers like Decagon, 1x, and Runway use Clusters for trillion-parameter fine-tuning, pre-training home robot world models, and multi-node video inference respectively. Clusters are available to all workspaces today through the @modal.clustered decorator, with per-second billing and no reservation required.
Approach / What changed
Modal Clusters provide a serverless, multi-node compute primitive for distributed AI workloads. The underlying technology includes a gang scheduler that observes the fleet, assigns all pending clusters to nodes, and provisions new capacity when needed. It also provides automated RDMA support (up to 6.4 Tbps per node) for PyTorch and NCCL, with the RDMA stack integrated into the gVisor container runtime. Users enable clusters with the @modal.clustered decorator and rdma=True flag, paying per-second with no reservation.
Takeaways
Modal Clusters provide a serverless multi-node compute primitive for distributed AI workloads, enabled by a single @modal.clustered decorator and a gang scheduler that places N nodes simultaneously.
RDMA support (up to 6.4 Tbps per node) is automated for PyTorch and NCCL behind the rdma=True flag, and Modal built and upstreamed RDMA support into the gVisor container runtime.
Customers pay per second with no per-hour reservation, and the same scheduler handles both multi-node clusters and single-node jobs, allowing users to burst to hundreds of GPUs and scale back down.
FAQ
How does Modal's gang scheduler work?
Modal Clusters use a gang scheduler that observes the entire fleet and all pending multi-node workloads in one iteration, decides a placement for each cluster, spins up new capacity if needed, and then sends schedule notifications to live nodes. This allows cluster acquisition in seconds from the same capacity pool as single-node workloads.
What is the pricing model for Modal Clusters?
Modal Clusters are billed by the second with no per-hour reservation. Users only pay for what they use, and can spin up hundreds of GPUs for a burst of experiments and scale back down between runs.
Source and freshness
- Source
- modal.com
- Published
- Added to Yomu
Summaries are generated from the original by a language model. Check the original before relying on it.
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
The best engineering reads, once a week
One useful email. Unsubscribe anytime.