---
title: "FSDP"
description: "1 posts about FSDP, summarised, each linking to the original."
---

# FSDP
> 1 posts about FSDP, summarised, each linking to the original.

## Articles

### [Integrating TorchTPU into TorchTitan and Optimizing Models for TPU](https://yomu.fyi/post/integrating-torchtpu-into-torchtitan-and-optimizing-models-for-tpu.md)
- Company: [Google](https://yomu.fyi/company/google.md)
- Author: Aleksey Vlasenko, Rob Torres, Will Constable
- Published: Oct 20, 2026
- Type: Announcement

TorchTitan TPU is a fork of Meta’s TorchTitan that enables PyTorch model training on Google Cloud TPUs using the TorchTPU native backend with minimal device initialization changes. SimpleFSDP replaces FSDP2 hooks with DTensor parametrization, making all-gathers and reduce-scatters ordinary nodes in the traced graph for compiler scheduling. The Graph Trainer captures the full forward, loss, and backward step as one XLA program via make\_fx, achieving 51-52% MFU on Qwen3-8B with functional AdamW. On Llama 3 405B, TorchTitan TPU reaches about 57% MFU on up to 1,024 chips, and a weak-scaling test of Qwen3 235B MoE retained 92% of per-chip throughput from 64 to 8,192 chips. The repository includes an optimization playbook, a coding-agent model-onboarding skill, and supports over 500 Hugging Face models through a Transformers backend.