# Kernel Fusion in NVIDIA CUDA: Optimizing Memory Traffic and Launch Overhead

[NVIDIA Developer Blog](https://yomu.fyi/company/nvidia-developer-blog) · Michelle Horton · Jul 10, 2026

**Type:** Explainer

## Summary

GPU compute often outpaces memory bandwidth, causing separate kernels to suffer from intermediate global memory round-trips and launch overhead. Kernel fusion resolves this by executing multiple operations within a single kernel to keep intermediate data inside registers. Using the sum(abs(x)) operation as a benchmark on an NVIDIA GeForce RTX 4090, a naive two-kernel setup moved 3 GB of global memory over 3.51 ms. In contrast, manual CUDA C++ fusion, PyTorch compiler fusion via torch.compile, and explicit Python fusion using cuda.compute all achieved a 3x speedup. Across all three approaches, global memory transfers dropped to 1 GB by computing absolute values inline during reduction.

## Context

GPU compute speed frequently outpaces device memory bandwidth, causing un-fused kernels to waste bandwidth on intermediate global memory round-trips and incur separate kernel launch overheads.

## Approach / What changed

Fusing operations such as element-wise transforms and reductions into single kernels keeps intermediate data in registers. This can be implemented via manual CUDA C++ with CCCL and CUB, implicit compiler fusion via torch.compile, or explicit Python composition using cuda.compute and TransformIterator.

## Takeaways

- Manual CUDA C++ kernel fusion reduced global memory traffic from 3 GB to 1 GB and execution time from 3.51 ms to 1.18 ms on an RTX 4090 for sum(abs(x)).
- CUDA Graphs capture dispatches and reduce host launch overhead, but they do not fuse kernel bodies or eliminate global memory round-trips for intermediate buffers.
- cuda.compute brings CUB-backed iterator-based explicit fusion to Python, avoiding intermediate allocations without requiring custom CUDA C++ kernels.

**Tags:** [Performance](https://yomu.fyi/topic/performance), [Python](https://yomu.fyi/topic/python)

[Read original post](https://developer.nvidia.com/blog/kernel-fusion-in-nvidia-cuda-optimizing-memory-traffic-and-launch-overhead)