# ModelExpress: Distributing Model Artifacts at the Speed of Light

[NVIDIA Developer Blog](https://yomu.fyi/company/nvidia-developer-blog) · Elizabeth Goodman · Jul 24, 2026

**Type:** Problem & solution

## Summary

Distributing large language model weights and JIT kernel caches across clusters imposes high latency on cold starts, autoscaling, and reinforcement learning loops. NVIDIA ModelExpress minimizes this delay by identifying existing compatible weight replicas across the cluster rather than treating every worker as an independent cold start. Initial replicas bootstrap by streaming directly from remote object storage into GPU memory or bypassing host staging using GPUDirect Storage. Subsequent replicas load pre-processed weights and kernel caches directly from serving peers over peer-to-peer RDMA via the NVIDIA Inference Xfer Library. In testing on DeepSeek-V4 Pro, this peer-to-peer artifact distribution transferred weights and caches in under ten seconds, reducing overall replica startup time from eight minutes to one minute and forty-four seconds.

## Context

Moving model checkpoints spanning hundreds of gigabytes or terabytes creates significant latency across cold starts, rolling deployments, autoscaling replicas, and reinforcement learning rollout updates. Furthermore, JIT kernel compilation and autotuning add several minutes of warmup latency on fresh replicas.

## Approach / What changed

NVIDIA ModelExpress dynamically routes artifact loading through the fastest available path. When bootstrapping, it pipelines reads directly from object storage to GPU memory or uses GPUDirect Storage, while collapsing duplicate cluster downloads. For subsequent workers, it bypasses storage entirely by using peer-to-peer RDMA via NIXL to transfer resident weights, JIT kernel caches, and RL step updates directly between peer GPUs.

## Takeaways

- MX transfers DeepSeek-V4 Pro weights and JIT kernel cache artifacts between replicas in under 10 seconds, cutting overall startup time from 8 minutes to 1 minute 44 seconds.
- For initial bootstrap from object storage, Model Streamer pipelines concurrent tensor reads directly into GPU memory via bounded host staging without writing to local disk.
- For RL post-training, MX uses a receiver-driven refit model where rollout workers discover published trainer tensors and pull directly via one-sided reads into their target layouts.

**Tags:** [Architecture](https://yomu.fyi/topic/architecture), [Kubernetes](https://yomu.fyi/topic/kubernetes), [LLMs](https://yomu.fyi/topic/llm), [Machine Learning](https://yomu.fyi/topic/machine-learning), [Performance](https://yomu.fyi/topic/performance)

[Read original post](https://developer.nvidia.com/blog/modelexpress-distributing-model-artifacts-at-the-speed-of-light)