---
title: "How to use Google microbenchmarks for evaluating TPU performance"
description: "Evaluating TPU performance requires measurements beyond product specifications, because real-world behavior depends on architecture and workload constraints. The suite measures network collectives, compute, HBM bandwidth, host transfers, and Ragged-Paged Attention primitives, using metrics such as GB/s, latency, TFLOPs, MFU, and TTFT-related throughput. Its src and configs directories provide implementations and tunable parameters, while sample Kubernetes recipes connect measurements to TPU architecture and runtime behavior. Roofline analysis uses peak MXU throughput and HBM bandwidth to distinguish compute-, memory-, and network-bound workloads and guide kernel, sharding, or rematerialization choices. In a 110B MoE workload on a 4x4x4 TPU 7x configuration, microbenchmark-guided changes addressed ICI stalls and memory-bound attention, reducing training step time by 21.2%."
---

# How to use Google microbenchmarks for evaluating TPU performance

[Google](https://yomu.fyi/company/google) · Junjie Qian, Chi Shuen Lee, Yu-Hsuan (Amy) Lin, Haixiong (Sean) Wang · Jul 30, 2026

**Type:** Explainer

## Summary

Evaluating TPU performance requires measurements beyond product specifications, because real-world behavior depends on architecture and workload constraints. The suite measures network collectives, compute, HBM bandwidth, host transfers, and Ragged-Paged Attention primitives, using metrics such as GB/s, latency, TFLOPs, MFU, and TTFT-related throughput. Its src and configs directories provide implementations and tunable parameters, while sample Kubernetes recipes connect measurements to TPU architecture and runtime behavior. Roofline analysis uses peak MXU throughput and HBM bandwidth to distinguish compute-, memory-, and network-bound workloads and guide kernel, sharding, or rematerialization choices. In a 110B MoE workload on a 4x4x4 TPU 7x configuration, microbenchmark-guided changes addressed ICI stalls and memory-bound attention, reducing training step time by 21.2%.

## Context

Product specifications and theoretical throughput do not fully describe how TPUs perform within particular architectural environments and workload constraints. Granular measurements are needed to identify performance gaps, validate hardware behavior, and locate bottlenecks such as VMEM locality, interconnect efficiency, HBM saturation, and host-transfer limits.

## Approach / What changed

The microbenchmark suite evaluates network collectives, compute, HBM, host transfers, and Ragged-Paged Attention through implementations in src/ and tunable configurations in configs/. It uses metrics including bandwidth, latency, TFLOPs, MFU, and attention throughput, then applies Speed-of-Light baselines and Roofline analysis to guide kernel selection, sharding, communication overlap, and rematerialization. A 110B MoE workload on a 4x4x4 TPU 7x configuration demonstrates the method.

## Takeaways

- Network tests cover all-gather, all-reduce, reduce-scatter, and all-to-all, with throughput and latency used to characterize ICI collective performance.
- The configs directory exposes buffer sizes, iteration counts, and supported data types including bf16, fp8, and fp32, allowing benchmark runs to be tuned.
- In the case study, SparseCore collective offloading addressed communication stalls, while Tokamax Splash Attention targeted HBM-bound attention primitives.

**Tags:** [Architecture](https://yomu.fyi/topic/architecture), [Performance](https://yomu.fyi/topic/performance), [TPU](https://yomu.fyi/topic/tpu)

- Source: [Google](https://developers.googleblog.com/how-to-use-google-microbenchmarks-for-evaluating-tpu-performance)
- Source URL: https://developers.googleblog.com/how-to-use-google-microbenchmarks-for-evaluating-tpu-performance
- Ingested by Yomu: 2026-09-09T14:28:13.317Z

[Read original post](https://developers.googleblog.com/how-to-use-google-microbenchmarks-for-evaluating-tpu-performance)