---
title: "To Infinity and Beyond: ThunderKittens Now on NVIDIA Vera Rubin NVL72!"
description: "Together’s kernels team ported ThunderKittens to NVIDIA Vera Rubin NVL72 and investigated why existing Blackwell GEMMs underuse the platform. The work targets NVFP4 and FP8 GEMMs: although Rubin preserves Blackwell’s programming model, the original kernels reach only about 42.1% and 44.4% of roofline because tensor cores consume operands twice as fast as the old pipeline supplies them. The team integrates Rubin’s 64-byte K step, exclusive tensor memory extending to 576 columns, oversized shared memory up to 328 KiB, B-side collector buffers, and earlier A-release signaling into its NVFP4 kernel. The resulting kernel exceeds 22 PFLOPS and is described as competitive with cuBLAS and CuTe DSL, while the post also compares Rubin’s specifications with NVIDIA HGX B200."
---

# To Infinity and Beyond: ThunderKittens Now on NVIDIA Vera Rubin NVL72!

[Together AI](https://yomu.fyi/company/together-ai) · Sep 10, 2026

**Type:** Explainer

## Summary

Together’s kernels team ported ThunderKittens to NVIDIA Vera Rubin NVL72 and investigated why existing Blackwell GEMMs underuse the platform. The work targets NVFP4 and FP8 GEMMs: although Rubin preserves Blackwell’s programming model, the original kernels reach only about 42.1% and 44.4% of roofline because tensor cores consume operands twice as fast as the old pipeline supplies them. The team integrates Rubin’s 64-byte K step, exclusive tensor memory extending to 576 columns, oversized shared memory up to 328 KiB, B-side collector buffers, and earlier A-release signaling into its NVFP4 kernel. The resulting kernel exceeds 22 PFLOPS and is described as competitive with cuBLAS and CuTe DSL, while the post also compares Rubin’s specifications with NVIDIA HGX B200.

## Context

NVIDIA Vera Rubin preserves the Blackwell programming model but provides higher vendor-specified tensor-core performance, memory bandwidth, SM count, tensor memory capacity, and optional shared memory. The post frames these changes around GEMM implementation and compares Vera Rubin NVL72 with NVIDIA HGX B200.

## Approach / What changed

The team starts with an existing Blackwell GEMM and incrementally integrates Vera Rubin features into its NVFP4 implementation. The changes use the 64-byte K step, exclusive tensor-memory allocation, oversized shared memory, B-side collector qualifiers, and the tensor\_aread\_commit operation for earlier A-tile release.

## Takeaways

- On Vera Rubin, the existing NVFP4 and FP8 kernels reach about 42.1% and 44.4% of roofline because the tensor-core pipeline consumes operands faster than the Blackwell kernel supplies them.
- Vera Rubin adds a 64-byte K step, up to 576 columns of exclusive tensor memory, up to 328 KiB of oversized shared memory, B-side collector support, and earlier A-tile release signaling.
- Integrating the new features takes the NVFP4 GEMM to over 22 PFLOPS, with performance described as competitive with cuBLAS and CuTe DSL.

## FAQ

### Why do the original Blackwell GEMMs underperform on Vera Rubin?

Vera Rubin’s tensor cores can consume operands twice as fast, while the old Blackwell kernel does not feed them quickly enough. The original NVFP4 and FP8 kernels therefore reach only about 42.1% and 44.4% of roofline.

### Which Vera Rubin features does the optimized GEMM use?

The implementation uses a 64-byte K step, exclusive tensor memory with up to 576 columns, oversized shared memory up to 328 KiB, B-side collector buffers, and earlier signaling when MMA finishes reading A.

**Tags:** [CUDA](https://yomu.fyi/topic/cuda), [GPU](https://yomu.fyi/topic/gpu), [NVIDIA](https://yomu.fyi/topic/nvidia), [Optimization](https://yomu.fyi/topic/optimization)

- Source: [Together AI](https://www.together.ai/blog/to-infinity-and-beyond-thunderkittens-now-on-nvidia-vera-rubin-nvl72)
- Source URL: https://www.together.ai/blog/to-infinity-and-beyond-thunderkittens-now-on-nvidia-vera-rubin-nvl72
- Ingested by Yomu: 2026-09-19T10:13:47.717Z

[Read original post](https://www.together.ai/blog/to-infinity-and-beyond-thunderkittens-now-on-nvidia-vera-rubin-nvl72)