How flat is replacing fat in AWS data center networks

Summary

Traditional data center networks rely on hierarchical fat-tree topologies, which introduce overhead, upper-layer congestion, and vulnerability to router failures. Although flat random networks theoretically maximize route diversity and resilience, routing computation and cabling complexity historically made them impractical for physical data centers. AWS addressed these constraints by introducing Resilient Network Graphs, a quasi-random architecture powered by a passive optical device called ShuffleBox and a routing algorithm named Spraypoint. Spraypoint sprays packets across random neighbors toward designated waypoints, unlocking twice as many independent paths while operating within commodity router memory limits. Adopted as the default architecture across new AWS facilities, this design reduces router counts by 69%, lowers projected network power consumption by 40%, and increases throughput by up to 33%.

Takeaways

1. 1

   Resilient Network Graphs reduce the number of routers by 69%, achieve up to 33% higher throughput, and project a 40% reduction in network equipment electricity consumption compared to fat trees.

2. 2

   The Spraypoint routing algorithm distributes traffic across random neighbors toward destination waypoints, delivering nearly twice as many independent paths as shortest-path routing on commodity router memory.

3. 3

   The ShuffleBox passive optical device standardizes physical cabling to match fat-tree installation simplicity while producing a quasi-random logical topology.