# Inside NVIDIA Rubin GPU Architecture: Powering the Era of Agentic AI

[NVIDIA Developer Blog](https://yomu.fyi/company/nvidia-developer-blog) · Eduardo Alvarez · Jul 21, 2026

**Type:** Explainer

## Summary

Agentic artificial intelligence workloads require sustained multistep inference, demanding low per-step latency, high decode throughput, and efficient long-context processing. NVIDIA designed the Rubin GPU to deliver up to ten times more agentic throughput per unit of energy than Blackwell. The hardware combines 336 billion transistors across two dies unified by NV-HBI, up to 288 GB of HBM4 memory providing 22 TB/s peak bandwidth, and the third-generation Transformer Engine reaching up to 50 petaflops of NVFP4 compute. Performance optimizations include inline descriptor updates for the Tensor Memory Accelerator, doubled K-dimension instruction throughput, 3-bit lookup-table weights, and structured 2:4 sparse attention compression. At the rack level, Vera Rubin NVL72 leverages Intelligent Power Smoothing and DSX MaxLPS to enable up to 40% more GPUs within a fixed power budget.

## Context

Agentic AI workloads require continuous reasoning, tool use, and long-context processing across many steps, challenging data centers to provide high throughput and low latency within fixed power envelopes.

## Approach / What changed

NVIDIA created the Rubin GPU and Vera Rubin NVL72 rack architecture, featuring dual-die packaging, HBM4 memory, expanded precision support, inline TMA updates, doubled K-dimension GEMM throughput, structured attention sparsity, and rack-scale power smoothing.

## Takeaways

- The Rubin GPU integrates two dies over NV-HBI, providing 336 billion transistors, 224 streaming multiprocessors, 896 Tensor Cores, and up to 288 GB of HBM4 memory with 22 TB/s bandwidth.
- Matrix operations gain efficiency by doubling Tensor Core throughput along the K dimension, which reduces loop iterations and benefits context and decode GEMMs at high tensor-parallel scale.
- Vera Rubin NVL72 uses Intelligent Power Smoothing and DSX MaxLPS to absorb transient power swings, enabling operators to provision up to 40% more GPUs within the same power budget.

**Tags:** [Architecture](https://yomu.fyi/topic/architecture), [LLMs](https://yomu.fyi/topic/llm), [Performance](https://yomu.fyi/topic/performance), [Scalability](https://yomu.fyi/topic/scalability)

[Read original post](https://developer.nvidia.com/blog/inside-nvidia-rubin-gpu-architecture-powering-the-era-of-agentic-ai)