NVIDIA NVLink Fusion Brings NVHBM to Next-Generation AI Infrastructure

Summary

AI factories need custom XPUs to handle larger models and complex reasoning workloads, but scaling them requires HBM bandwidth, package and silicon area, power efficiency, supply-chain resilience, and rack-scale integration. NVIDIA NVLink Fusion addresses this by connecting custom XPUs and CPUs to NVIDIA’s AI infrastructure platform, using its scale-up and scale-out stack, ecosystem, and MGX rack-scale architecture. Its package-level NVHBM custom HBM base die, designed and validated with leading memory vendors, provides up to 30% more bandwidth and 15% lower HBM power usage than standard HBM4e, while enabling up to 25% more compute die area. The combined NVLink Fusion and NVHBM design is described as delivering a 30% overall end-to-end performance increase per XPU and simplifying integration into heterogeneous, rack-scale deployments.

Takeaways

1. 1

   NVHBM provides up to 30% more memory bandwidth per stack and up to 15% lower HBM power usage than standard HBM4e, targeting bandwidth-intensive training and inference workloads.

2. 2

   NVHBM reduces PHY and support area by up to 67% and can provide up to 25% more compute die area; the source also reports up to 80% more usable silicon across the layout.

3. 3

   The NVLink Fusion chiplet connects custom XPUs across a rack-scale NVLink domain, supporting distributed compute, expert parallelism, synchronized caches, and integration with GPUs for heterogeneous compute.