Loading post…
Modernizing Artifact Storage at Uber
Summary
Uber’s legacy artifact platform used two five-node clusters with local disks, asynchronous replication, and GeoDNS failover to support builds across monorepos and microrepos. As artifact volume grew, limited node capacity, inconsistent replication, manual hardware evacuation, and risky multi-terabyte upgrades created operational and data-loss risks. Uber moved to a managed SaaS platform with multi-region availability, cross-region replication, and cloud-native blob storage, then added an internal validation proxy in front of it. The proxy uses MySQL metadata and conditional requests, including If-None-Match and If-Modified-Since, to validate artifacts against the SaaS origin while reusing unchanged bytes; failures can fall back to passthrough, preserving correctness at higher latency and egress cost. The design reduced operational risk, improved observability, and made artifact delivery more cost-efficient at scale.
Context
Uber’s artifact platform was on the critical path for builds spanning large monorepos and thousands of microrepos. The previous on-premises design relied on local disks, manual recovery, asynchronous replication, and coordinated upgrades, creating capacity, reliability, and operational risks. Moving artifact storage to a managed SaaS platform introduced significant egress costs because Uber downloads more than 5 petabytes of artifacts monthly.
Approach / What changed
Uber adopted a managed SaaS artifact platform with multi-region availability, asynchronous cross-region replication, and cloud-native blob storage. An internal validation proxy routes all artifact requests to the SaaS origin, records artifact URLs, checksums, and last-modified timestamps in MySQL, and uses conditional requests to avoid redownloading unchanged bytes. It can fall back to passthrough mode when internal dependencies fail, while active-active deployment and GeoDNS support availability across proxy instances, data centers, and SaaS regions.
Takeaways
Local-disk capacity limits, under-replication, manual node evacuation, and multi-terabyte upgrade migrations made the legacy platform difficult and risky to operate.
The validation proxy is not a TTL-based cache: it validates each request against the SaaS origin using conditional requests and serves cached bytes only after receiving confirmation that the artifact is unchanged.
Proxy and dependency failures degrade to higher latency or egress cost through passthrough and failover paths, rather than allowing stale, incomplete, or corrupted artifacts to be served.
FAQ
Why did Uber build a validation proxy instead of using a generic HTTP cache?
Generic cache semantics based on TTLs and best-effort eviction did not fit artifact correctness requirements. Uber needed per-request validation, strong consistency guarantees, and graceful passthrough on failures.
How does the proxy avoid repeatedly downloading unchanged artifacts?
It stores artifact metadata, including the URL, checksum, and last-modified timestamp, in MySQL. Conditional requests using If-None-Match and If-Modified-Since let the SaaS origin return 304 Not Modified, after which the proxy can serve cached bytes.
What happens when the proxy or its dependencies fail?
The proxy can switch to passthrough mode and forward requests directly to the SaaS origin without relying on local state. Builds continue to function, but with higher latency and egress costs.
Source and freshness
- Source
- uber.com
- Published
- Added to Yomu
Summaries are generated from the original by a language model. Check the original before relying on it.
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
The best engineering reads, once a week
One useful email. Unsubscribe anytime.