NVIDIA NVLink: The Scale-Up Network for AI Factories

Summary

Scale-up networking has emerged as a critical architectural component for AI factories supporting trillion-parameter models, mixture-of-experts architectures, and disaggregated serving. NVIDIA presents its sixth-generation NVLink scale-up fabric and NVLink 6 Switch to address latency-sensitive all-to-all GPU communication and offload collective operations. In simulation benchmarks on models like DeepSeek-R1, Qwen 235B, and a simulated 2T parameter model, NVLink achieved up to 2.3X higher decode throughput than off-the-shelf Ethernet in a 72-accelerator domain. The architecture integrates NVLink-C2C to supply 1.8 TB/s of coherent CPU-GPU bandwidth in the Vera Rubin NVL72 platform, as well as NVLink Fusion for integrating semi-custom silicon XPUs. Through co-designed hardware, SHARP in-network compute, and NCCL libraries, the platform optimizes token throughput, factory uptime, and deployment economics.

Takeaways

1. 1

   Sixth-generation NVLink delivers up to 2.3X higher decode throughput compared to off-the-shelf Ethernet in a 72-accelerator scale-up domain based on simulations of DeepSeek-R1, Qwen 235B, and a 2T parameter model.

2. 2

   NVLink-C2C provides 1.8 TB/s of coherent CPU-to-GPU bandwidth in the Vera Rubin NVL72 platform, representing 7x the bandwidth of PCIe Gen6.

3. 3

   NVLink Fusion allows third-party custom silicon XPUs to connect directly to the NVLink scale-up stack to reduce development complexity and integrate with GPUs.