Together AIAnnouncement
Introducing Together Storage Accelerator: fast, free data caching for your GPU cluster
Summary
Together Storage Accelerator (TSA) is a distributed caching layer for Together GPU Clusters that eliminates repeated data reads from object storage and Hugging Face Hub. It runs on every GPU node and presents drop-in endpoints for existing storage, requiring no application changes. The first read from any node caches data on cluster NVMe, so subsequent reads from any node are served locally. Measured on an H100 node, warm reads are about 20 times faster than cross-cloud downloads, reaching up to 3.4 GB/s per node, and a 70 GB checkpoint pulled by 200 nodes downloads from the source only once. TSA is included free with every Together GPU Cluster and is powered by Together’s ASTRA distributed caching technology.
Context
Together Storage Accelerator is a distributed caching layer for GPU clusters, included free with Together GPU Clusters. It addresses the problem of slow data reads from remote object storage and Hugging Face Hub by caching data on cluster NVMe, providing drop-in endpoints for S3-compatible storage and Hugging Face Hub without application changes. It is powered by ASTRA, Together’s distributed caching technology.
Approach / What changed
TSA runs on every GPU node in the cluster and presents drop-in endpoints for S3-compatible storage and Hugging Face Hub. The first read from any node caches the object on cluster NVMe; subsequent reads from any node are served from the cache. Authentication is handled with encrypted credentials stored in a secrets vault. Users enable it from the cluster Settings page or via a single command setting environment variables like AWS_ENDPOINT_URL or HF_ENDPOINT.
Takeaways
TSA provides ~20x faster warm reads than cross-cloud downloads, with up to 3.4 GB/s per node on H100.
A single origin fetch serves the entire cluster; a 70 GB checkpoint pulled by 200 nodes downloads from the source once instead of 200 times.
TSA requires no application changes; users set environment variables (e.g., AWS_ENDPOINT_URL) to point to the TSA endpoint.
FAQ
How do I enable Together Storage Accelerator?
Enable it from your cluster's Settings page or set environment variables like AWS_ENDPOINT_URL or HF_ENDPOINT to the TSA endpoint. No application changes are needed.
What storage backends does TSA support?
TSA works with any S3-compatible store, including Amazon S3, Cloudflare R2, and Together AI's T4, as well as Hugging Face Hub.
Is there any additional cost for TSA?
TSA is included free with every Together GPU Cluster. There is no additional cost.
Source and freshness
- Source
- together.ai
- Published
- Added to Yomu
Summaries are generated from the original by a language model. Check the original before relying on it.
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
The best engineering reads, once a week
One useful email. Unsubscribe anytime.