---
title: "Profiling in PyTorch (Part 1): A Beginner's Guide to torch.profiler"
description: "The post introduces torch.profiler through a beginner-oriented walkthrough of profiling a matrix multiplication and addition, explaining how to configure the profiler and interpret its outputs. It uses a small function with torch.matmul and torch.add, annotates it with record_function, captures CPU and CUDA activities after warm-up iterations, and exports a profiler table plus a Chrome trace. The table summarizes event counts and CPU or GPU time, while the trace shows execution order, call hierarchy, CPU-GPU gaps, kernel launches, and overlap. Comparing 64x64 and 4096x4096 matrices demonstrates a shift from overhead-bound behavior to compute-bound behavior; the guide also covers dispatch chains, hidden device-to-device copies, profiler buffer activity, and torch.compile's runtime and caching behavior."
---

# Profiling in PyTorch (Part 1): A Beginner's Guide to torch.profiler

[Hugging Face](https://yomu.fyi/company/hugging-face) · Aritra Roy Gosthipaty, Sayak Paul, Sergio Paniego, Rémi Ouazan Reboul, Pedro Cuenca · May 29, 2026

**Type:** Tutorial

## Summary

The post introduces torch.profiler through a beginner-oriented walkthrough of profiling a matrix multiplication and addition, explaining how to configure the profiler and interpret its outputs. It uses a small function with torch.matmul and torch.add, annotates it with record\_function, captures CPU and CUDA activities after warm-up iterations, and exports a profiler table plus a Chrome trace. The table summarizes event counts and CPU or GPU time, while the trace shows execution order, call hierarchy, CPU-GPU gaps, kernel launches, and overlap. Comparing 64x64 and 4096x4096 matrices demonstrates a shift from overhead-bound behavior to compute-bound behavior; the guide also covers dispatch chains, hidden device-to-device copies, profiler buffer activity, and torch.compile's runtime and caching behavior.

## Context

Profiling has a steep on-ramp because traces contain dense visual timelines and unfamiliar event names, while many tutorials assume prior experience reading them. The post aims to make profiling accessible to readers with only basic PyTorch knowledge and uses a small matrix multiplication and addition example to establish transferable mental models.

## Approach / What changed

The post wraps a matrix multiplication and addition function in torch.profiler.record\_function and torch.profiler.profile, enables CPU and CUDA activities, runs repeated iterations to warm up the GPU, and exports key\_averages().table output and a Chrome trace. It compares small and large matrix sizes, then explains profiler lanes, dispatch chains, CUDA kernels, hidden copies, and torch.compile traces.

## Takeaways

- For 64x64 matrices, GPU work takes less than 1% of CPU time because kernel preparation, launches, and data movement dominate; the example is overhead-bound.
- Increasing the matrix size to 4096 makes self CPU and CUDA totals comparable, with the GPU GEMM kernel taking most CUDA time and the workload becoming compute-bound.
- In torch.compile traces, Dynamo cache lookup and the AOTAutograd-to-Inductor runtime stack remain visible on each call, so compilation overhead may exceed eager execution time for tiny operations.

**Tags:** [Machine Learning](https://yomu.fyi/topic/machine-learning), [Observability](https://yomu.fyi/topic/observability), [Performance](https://yomu.fyi/topic/performance), [Python](https://yomu.fyi/topic/python)

- Source: [Hugging Face](https://huggingface.co/blog/torch-profiler)
- Source URL: https://huggingface.co/blog/torch-profiler
- Ingested by Yomu: 2026-08-29T01:49:32.423Z

[Read original post](https://huggingface.co/blog/torch-profiler)