---
title: "NVIDIA Vera Rubin and Blackwell Set a New Standard for Agentic AI Performance per Watt"
description: "Agentic AI inference has shifted from single-turn requests to long, stateful workflows that reason, invoke tools, coordinate subagents, and reuse growing context, increasing token and power demands. The post presents SemiAnalysis AgentX, an InferenceX benchmark that replays prerecorded Claude Code sessions turn by turn, preserving context, input/output lengths, reasoning time, tool-call latency, and KV-cache pressure while varying concurrency. It evaluates tokens per megawatt alongside E2E normalized interactivity, standard interactivity, E2E latency, and TTFT; NVIDIA-measured Vera Rubin NVL72 results, pending SemiAnalysis review, show up to 30x higher throughput per megawatt than GB300 NVL72 at 160 tokens per second per user. GB300 also delivers up to 15x H200 throughput per megawatt, up to 10x lower cost per million tokens, and roughly 80x higher throughput on Kimi K3 2.8T, supported by MoE runtimes, communication overlap, Dynamo, and NVLink."
---

# NVIDIA Vera Rubin and Blackwell Set a New Standard for Agentic AI Performance per Watt

[NVIDIA](https://yomu.fyi/company/nvidia-developer-blog) · Elizabeth Goodman · Aug 24, 2026

**Type:** Benchmark

## Summary

Agentic AI inference has shifted from single-turn requests to long, stateful workflows that reason, invoke tools, coordinate subagents, and reuse growing context, increasing token and power demands. The post presents SemiAnalysis AgentX, an InferenceX benchmark that replays prerecorded Claude Code sessions turn by turn, preserving context, input/output lengths, reasoning time, tool-call latency, and KV-cache pressure while varying concurrency. It evaluates tokens per megawatt alongside E2E normalized interactivity, standard interactivity, E2E latency, and TTFT; NVIDIA-measured Vera Rubin NVL72 results, pending SemiAnalysis review, show up to 30x higher throughput per megawatt than GB300 NVL72 at 160 tokens per second per user. GB300 also delivers up to 15x H200 throughput per megawatt, up to 10x lower cost per million tokens, and roughly 80x higher throughput on Kimi K3 2.8T, supported by MoE runtimes, communication overlap, Dynamo, and NVLink.

## Context

Fixed-length serving scenarios are less representative of production agent sessions, where request lengths vary, context accumulates, previously processed tokens can be reused, and model calls are interrupted by tool execution or delegated work. AgentX addresses the need to measure long-context prefill, KV-cache reuse, interactive decode, tool-call gaps, distributed MoE execution, concurrency, and power efficiency under realistic agentic traffic.

## Approach / What changed

AgentX replays prerecorded Claude Code sessions with the AIPerf client, turn by turn, using identical recorded traffic for each system. It preserves each session's context, input/output lengths, reasoning time, tool-call latency, and timing intervals, then varies concurrency to measure throughput per provisioned megawatt against E2E normalized interactivity, standard interactivity, E2E latency, and TTFT. The described GB300 serving stack uses MoE runtimes and kernels, communication-computation overlap, Dynamo's prefill/decode separation and KV-cache-aware routing, and NVLink scale-up.

## Takeaways

- AgentX preserves recorded agent-session timing, context growth, reasoning intervals, and tool-call latency, allowing KV-cache-capacity pressure and repeated-context reuse to affect the measurement.
- AgentX reports tokens per megawatt alongside E2E normalized interactivity, standard interactivity, E2E latency, and TTFT, linking power efficiency with user-visible responsiveness.
- NVIDIA-measured Vera Rubin NVL72 results are pending SemiAnalysis review and show up to 30x higher throughput per megawatt than GB300 NVL72 at 160 tokens per second per user.

**Tags:** [Architecture](https://yomu.fyi/topic/architecture), [Performance](https://yomu.fyi/topic/performance), [Scalability](https://yomu.fyi/topic/scalability)

- Source: [NVIDIA](https://developer.nvidia.com/blog/nvidia-vera-rubin-and-blackwell-set-a-new-standard-for-agentic-ai-performance-per-watt)
- Source URL: https://developer.nvidia.com/blog/nvidia-vera-rubin-and-blackwell-set-a-new-standard-for-agentic-ai-performance-per-watt
- Ingested by Yomu: 2026-08-29T01:38:44.893Z

[Read original post](https://developer.nvidia.com/blog/nvidia-vera-rubin-and-blackwell-set-a-new-standard-for-agentic-ai-performance-per-watt)