# How to Optimize Transformer-Based Models for Low-Precision Training

[NVIDIA Developer Blog](https://yomu.fyi/company/nvidia-developer-blog) · Jonathan Mitchell · Jun 16, 2026

**Type:** Benchmark

## Summary

Accelerating transformer training requires understanding how low-precision formats such as FP8 and NVFP4 impact specific general matrix multiplication (GEMM) workloads. Transformer configurations do not explicitly reveal these shapes, making microbenchmarks necessary before committing to full training runs. NVIDIA Transformer Engine enables quantization and kernel dispatch across precisions, which can be evaluated in realistic autocast mode or kernel-only prequantized mode. Profiling the ESM2-15B model on NVIDIA B300 GPUs demonstrated that NVFP4 achieved a 1.79x blended forward propagation speedup over MXFP8 and up to 4.01x over BF16 in prequantized execution. Although large GEMM dimensions successfully overcome quantization overheads, dynamic scaling, Hadamard transforms, and kernel selection asymmetries moderate real-world gains.

## Context

Large transformer models demand significant GPU hours and engineering time during training. While GPUs support low-precision formats such as FP8 and NVFP4, transformer configurations obscure exact runtime GEMM shapes, complicating precision selection and overhead estimation.

## Approach / What changed

Using a microbenchmark tool with NVIDIA Transformer Engine, developers map transformer hyperparameters and batch sizes into exact M×K×N GEMM shapes for Fprop, Dgrad, and Wgrad stages, evaluating performance across precisions in autocast and prequantized modes on NVIDIA B300 GPUs.

## Takeaways

- Prequantized microbenchmarking isolates raw kernel speedups from dynamic quantization overheads, showing NVFP4 versus BF16 jumping from 2.69x in autocast mode to 4.01x in kernel-only execution.
- FP8 DelayedScaling proved to be the fastest FP8 recipe on Blackwell at 23.76 ms per layer in autocast mode, outperforming FP8 CurrentScaling and MXFP8 by leveraging an amax-history approach.
- Quantized Dgrad passes run 4 to 5 percent slower than Fprop passes due to matrix aspect ratio changes that affect kernel selection, making the standard 2x Fprop timing approximation inaccurate.

**Tags:** [LLMs](https://yomu.fyi/topic/llm), [Machine Learning](https://yomu.fyi/topic/machine-learning), [Performance](https://yomu.fyi/topic/performance)

[Read original post](https://developer.nvidia.com/blog/how-to-optimize-transformer-based-models-for-low-precision-training)