Loading…
Topic
Pathways
1 post about Pathways. Every summary links to the original.
- 1 post
- 1 company
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Posts
What is Pathways? Single-controller training for large models
Large-model training with SPMD struggles when jobs become non-uniform, such as pipeline parallelism or RL training loops, and cannot survive hardware failures without restarting. Pathways addresses this by introducing a single Python client that describes the entire computation as a graph of blocks and tensor edges. A distributed runtime places blocks on virtual devices, moves and reshards tensors, orders operations across the cluster, and remaps around a failed host. This separates model building from execution, allowing reasoning about the whole cluster instead of one slice. Pathways runs JAX programs on TPUs and is available on Google Cloud.
Related topics
All topics →Weekly digest
The best engineering reads, once a week.