Loading post…
NVIDIATanya LenzAnnouncement
NVIDIA Run:ai Is Now Open Source as Part of NVIDIA DSX AI Factory Platform
Summary
NVIDIA has released the complete Run:ai platform as open source software, expanding the NVIDIA DSX AI Factory Platform and giving customers, developers, and ecosystem partners production-grade AI workload and GPU orchestration capabilities. Run:ai has dynamically allocated GPU resources and managed workloads on Kubernetes across data processing, training, and inference. The release includes the full platform plus four independently adoptable components: KAI Scheduler, GPU Fractioning Operator, KAI Resource Management, and Workload Map. Teams can deploy the stack from the NVIDIA Run:ai repository in the DSX AI Factory GitHub organization or via an NGC Community Product Helm chart, without a paid subscription or enterprise support entitlement. Ecosystem partners including Crusoe, Mirantis, Nebius, OpenNebula Systems, Rafay, Saturn Cloud, Spectro Cloud, and vCluster are integrating the platform or its components into their offerings.
Context
NVIDIA Run:ai has helped organizations run AI workloads on Kubernetes at scale by dynamically allocating GPU resources and managing workloads across data processing, training, and inference. The open source release expands the NVIDIA DSX AI Factory Platform and establishes Run:ai as an open foundation for ecosystem partners. Partners can adopt the complete platform or individual components, including KAI Scheduler, GPU Fractioning Operator, KAI Resource Management, and Workload Map, while NVIDIA-supported commercial offerings remain available for NVIDIA Enterprise AI Factory deployments.
Approach / What changed
NVIDIA released the full Run:ai platform as open source software in the NVIDIA DSX AI Factory GitHub organization. The release includes the complete platform plus four independently adoptable components: KAI Scheduler, GPU Fractioning Operator, KAI Resource Management, and Workload Map. The integrated stack is available on NVIDIA NGC as a Community Product with a Helm chart. Teams can deploy it directly, build on it, or integrate components. NVIDIA DSX ecosystem partners can adopt the complete platform or individual components in their own AI factory solutions; the post lists Crusoe, Mirantis, Nebius, OpenNebula Systems, Rafay, Saturn Cloud, Spectro Cloud, and vCluster integrations.
Takeaways
KAI Scheduler was released under Apache 2.0 and contributed to CNCF as a Sandbox project; all scheduler development happens in the community repository, and it remains the upstream engine behind NVIDIA Run:ai.
GPU Fractioning Operator brings CUDA-native GPU fractioning to Kubernetes, allowing fractions to be requested on demand and set up on the fly without per-node administrator setup, with hard memory isolation between workloads.
Model Streamer is integrated upstream in vLLM and SGLang via the –load-format runai_streamer flag; Azure benchmarks on eight NVIDIA H100 VMs show 3x-6x faster loading and a steady 80 Gbps versus approximately 40 Gbps.
FAQ
Which parts of the Run:ai open source release can be adopted independently?
KAI Scheduler, GPU Fractioning Operator, KAI Resource Management, and Workload Map can be adopted independently. Together with additional platform code, they form the integrated NVIDIA Run:ai platform. Teams can also deploy the complete platform as a whole.
What licensing and support terms apply to the NGC Community Product?
The source code is governed by its applicable open source licenses. Downloading the Community Product from NGC requires acceptance of the applicable NVIDIA Software License Agreement and Product-Specific Terms for NVIDIA AI Products. No paid Run:ai subscription or NVIDIA support entitlement is required, and NVIDIA enterprise support is not included.
How is Model Streamer enabled in inference engines?
Model Streamer is integrated upstream in vLLM and SGLang, where it can be enabled with the --load-format runai_streamer flag, as well as in ModelExpress. Support covers local filesystems, S3-compatible stores, and network file shares alongside several cloud backends. Azure benchmarks on eight NVIDIA H100 VMs show 3x-6x faster loading and a steady 80 Gbps versus approximately 40 Gbps.
Source and freshness
- Source
- developer.nvidia.com
- Published
- Added to Yomu
Summaries are generated from the original by a language model. Check the original before relying on it.
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
The best engineering reads, once a week
One useful email. Unsubscribe anytime.