---
title: "How NVIDIA Groq 3 LPX Unlocks Ultrafast Interactivity at Long Context on NVIDIA Vera Rubin"
description: "Serving agentic AI at more than 3,000 output tokens per second while retaining a 100K-token KV cache creates a small-batch tensor-parallelism challenge: fixed communication and coordination costs can offset distributed-compute gains. NVIDIA Groq 3 LPX addresses this with compiler-scheduled execution across 256 LP30 local processing units, 128 GB of collective SRAM, and 96 112-Gbps C2C links per chip, planning transfers before execution and overlapping communication with computation at 320-byte-vector granularity. Artificial Analysis measured 3,431 output tokens per second at 100K context on Gemma 4 31B, and 3,382 tokens per second at 10K context. On SPEED-Bench coding tasks, the same system reached a median 4,767 tokens per second and a P80 of 5,520. The post describes pairing LPX with Vera Rubin NVL72 through prefill-decode, attention-FFN, or external-drafter speculative-decoding configurations for projected large-model agentic workloads."
---

# How NVIDIA Groq 3 LPX Unlocks Ultrafast Interactivity at Long Context on NVIDIA Vera Rubin

[NVIDIA](https://yomu.fyi/company/nvidia-developer-blog) · Tanya Lenz · Aug 24, 2026

**Type:** Explainer

## Summary

Serving agentic AI at more than 3,000 output tokens per second while retaining a 100K-token KV cache creates a small-batch tensor-parallelism challenge: fixed communication and coordination costs can offset distributed-compute gains. NVIDIA Groq 3 LPX addresses this with compiler-scheduled execution across 256 LP30 local processing units, 128 GB of collective SRAM, and 96 112-Gbps C2C links per chip, planning transfers before execution and overlapping communication with computation at 320-byte-vector granularity. Artificial Analysis measured 3,431 output tokens per second at 100K context on Gemma 4 31B, and 3,382 tokens per second at 10K context. On SPEED-Bench coding tasks, the same system reached a median 4,767 tokens per second and a P80 of 5,520. The post describes pairing LPX with Vera Rubin NVL72 through prefill-decode, attention-FFN, or external-drafter speculative-decoding configurations for projected large-model agentic workloads.

## Context

Agentic sessions repeatedly feed the growing outputs and accumulated knowledge from earlier turns back into the model. Context can reach hundreds of thousands of tokens, making long-context processing important, while serving more than 3,000 tokens per second with a 100K-token KV cache creates coordination challenges for tensor parallelism at very small batch sizes.

## Approach / What changed

Groq 3 LPX uses a deterministic, compiler-scheduled execution model with visibility into its LPUs, SRAM, and C2C links. It preschedules transfers to reduce first-bit latency and overlaps communication with computation at 320-byte-vector granularity. Paired with Vera Rubin NVL72, it supports prefill-decode disaggregation, attention-FFN disaggregation, and external-drafter speculative decoding.

## Takeaways

- Artificial Analysis measured 3,431 output tokens per second on Gemma 4 31B at 100K context and 3,382 output tokens per second at 10K context.
- Groq 3 LPX uses point-to-point C2C links, compiler-planned routing, and LPU-level scheduling to minimize transfer initialization and contention.
- On SPEED-Bench coding tasks, the system achieved a median of 4,767 output tokens per second and a P80 of 5,520 tokens per second.

**Tags:** [Architecture](https://yomu.fyi/topic/architecture), [Machine Learning](https://yomu.fyi/topic/machine-learning), [Performance](https://yomu.fyi/topic/performance), [Scalability](https://yomu.fyi/topic/scalability)

- Source: [NVIDIA](https://developer.nvidia.com/blog/how-nvidia-groq-3-lpx-unlocks-ultrafast-interactivity-at-long-context-on-nvidia-vera-rubin)
- Source URL: https://developer.nvidia.com/blog/how-nvidia-groq-3-lpx-unlocks-ultrafast-interactivity-at-long-context-on-nvidia-vera-rubin
- Ingested by Yomu: 2026-08-29T01:38:19.713Z

[Read original post](https://developer.nvidia.com/blog/how-nvidia-groq-3-lpx-unlocks-ultrafast-interactivity-at-long-context-on-nvidia-vera-rubin)