SalesforceScott NybergProblem & solution
Salesforce’s Cloud Atlas team operates a globally distributed identity data store that must maintain five nines of availability, roughly five minutes of downtime per year. The team redesigned service protection after per-instance rate limiting proved insufficient under autoscaling and increasingly bursty, unpredictable agent-driven traffic. The new approach combines intelligent load shedding, which monitors queue time as an early indicator of system pressure before error rates spike, with coordinator-free global quota management. Servers independently measure and share customer usage to throttle noisy tenants without a centralized coordinator, preventing one customer’s surge from affecting others. This design prevents cascading failures by shedding lower-priority work before overload spreads across dependent services, ensuring graceful degradation rather than widespread outage.
Cloud Atlas is a globally distributed identity data store that powers a large portion of Salesforce authentication, requiring five nines availability. The team faced challenges as per-instance rate limiting became outdated with autoscaling, and bursty agent-driven traffic created noisy-neighbor problems. Localized overload could cascade through retries, affecting dependent services.
The team implemented intelligent load shedding that monitors queue time to estimate system pressure and sheds lower-priority work before error rates spike. They also deployed coordinator-free global quota management where servers independently measure and share customer usage to throttle noisy tenants without a centralized coordinator, allowing the system to adapt as servers scale.
Per-instance rate limiting becomes ineffective under autoscaling because each server’s limits become outdated and busy servers reject requests while capacity remains elsewhere.
Monitoring queue time as an early indicator of system pressure allows load shedding to react before error rates spike, preventing cascading failures from retries.
Coordinator-free global quota management prevents noisy tenants from consuming shared capacity by having servers independently measure and share customer usage to reach throttling decisions.
Per-instance rate limiting became outdated as instances were added or removed, and each server only understood its own workload. Busy servers could reject requests even while capacity remained available elsewhere, and it could not prevent noisy tenants from consuming shared resources.
The system monitors queue time as an early indicator of system pressure. Latency rises before requests begin failing, so load shedding can react while the system is slowing down, shedding lower-priority work before cascading failures begin.
The team avoided centralized rate limiting because it would introduce another critical dependency. Instead, they used coordinator-free global quota management where servers independently reach throttling decisions by sharing customer usage information, ensuring no single coordinator becomes a bottleneck.
Summaries are generated from the original by a language model. Check the original before relying on it.
See the idea
Buildkite splits CI into a hosted control plane and agents you run on your own machines. This comparison weighs that self-hosted agent model against fully hosted runners so you can pick the right fleet.
Bot management is an architecture, not a rule list: classify traffic, score it with continuously retrained models, enforce with challenges or blocks, then feed the outcomes back into the model.
A Bloom filter is a small bit array plus a few hash functions that answers "has this been seen?" with no false negatives and a false-positive rate you choose up front.
bfcache freezes page snapshots for instant back/forward navigation, but unload listeners, cache headers, or third-party scripts can silently disable it. This explainer maps the trade-offs and the metrics that reveal them.
The API gateway is the single boundary where protocol translation, auth, quotas, and routing become shared policy instead of per-service code. This layered diagram shows what belongs there and why.
Keep reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
One useful email. Unsubscribe anytime.
