NVIDIA Vera CPU Boosts AI Factory Throughput to Accelerate Agentic Workloads

Summary

Agentic AI workflows and reinforcement learning rely heavily on sequential CPU tasks, such as tool calls, sandboxed code execution, data processing, and environment rollouts between GPU steps. When CPU performance degrades under full socket load, training time inflates, serving latency grows, and GPU KV-caches face eviction. The NVIDIA Vera CPU addresses these bottlenecks by featuring 88 Olympus cores on a monolithic compute die, paired with power-efficient LPDDR5x memory and the Scalable Coherency Fabric. These cores incorporate a neural branch predictor, a 10-wide decode front end, deep out-of-order execution, and NVIDIA Spatial Multithreading. Consequently, Vera achieves 1.8x faster core execution, 40% lower peak loaded latency compared to x86 CPUs, and up to 1.2 TB/s total memory bandwidth, preventing costly KV-cache recomputations.

Context

Agentic systems and reinforcement learning workflows require sequential, logic-based CPU execution between GPU model steps, causing fleet-wide training stalls, increased serving latency, and GPU KV-cache evictions when CPUs slow down under socket load.

Approach / What changed

NVIDIA created the Vera CPU with 88 Olympus cores on a monolithic compute die, utilizing LPDDR5x memory, the Scalable Coherency Fabric, a neural branch predictor, 10-wide decode front ends, deep out-of-order execution, and Spatial Multithreading.

Takeaways

• The Vera CPU uses 88 Olympus cores on a monolithic compute die and Scalable Coherency Fabric to achieve 40% lower peak loaded latency than x86 CPUs without cross-chiplet hops.
• With 1.8x faster cores, Vera CPU increases completed RL environment rollout evaluations within a training window from 45% on a baseline CPU up to 85%.
• LPDDR5x memory on the Vera CPU delivers up to 1.2 TB/s total memory bandwidth and up to 14 GB/s per core at less than half the power of traditional data center CPUs.

Related reading

• NVIDIA Developer Blog · Jul 21, 2026

  Inside NVIDIA Rubin GPU Architecture: Powering the Era of Agentic AI

  Agentic artificial intelligence workloads require sustained multistep inference, demanding low per-step latency, high decode throughput, and efficient long-context processing. NVIDIA designed the Rubin GPU to deliver up to ten times more agentic throughput per unit of energy than Blackwell. The hardware combines 336 billion transistors across two dies unified by NV-HBI, up to 288 GB of HBM4 memory providing 22 TB/s peak bandwidth, and the third-generation Transformer Engine reaching up to 50 petaflops of NVFP4 compute. Performance optimizations include inline descriptor updates for the Tensor Memory Accelerator, doubled K-dimension instruction throughput, 3-bit lookup-table weights, and structured 2:4 sparse attention compression. At the rack level, Vera Rubin NVL72 leverages Intelligent Power Smoothing and DSX MaxLPS to enable up to 40% more GPUs within a fixed power budget.

  Eduardo Alvarez

• NVIDIA Developer Blog · Jun 23, 2026