The Graviton Trade: Price-Performance vs. Migration Drag
AWS Graviton migration trades strong price-performance gains against real coordination costs. This map shows the benefit, the cost, and the conditions that decide when Arm adoption pays off.
MetaHaiyu LuAnnouncement
Meta Andromeda is Meta’s personalized ads retrieval engine, and its next generation uses model-system co-design to achieve near ranking-level performance at retrieval scale. The system scores over one billion ads per second, a 100x throughput increase over the 2024 system, and has delivered a 2.4% increase in conversions on Facebook since 2025. The key innovation is a retrieval mixture-of-experts (MoE) architecture that reduces inference-time FLOPs by ~300x compared to the 2024 model, enabling ranking-level matching quality. The entire retrieval pipeline is fused into a single GPU kernel, eliminating memory bandwidth bottlenecks and achieving over 10 billion ads per second on a single NVIDIA GH200 Grace Hopper Superchip. The co-design approach, developed in collaboration with NVIDIA, positions the architecture to absorb more model complexity without proportional infrastructure growth.
Meta Andromeda is Meta’s personalized ads retrieval engine. The 2024 system achieved tens of millions of ads per second per GPU. With generative AI tools, the ad candidate pool grew to unprecedented scale, requiring a 100x throughput increase. The new system co-designs model, system, and hardware to achieve near ranking-level performance at retrieval scale.
The approach includes a retrieval mixture-of-experts (MoE) architecture with specialized user experts and an MoE attention-based gating network, fused GPU kernels that combine model and system workloads, and co-location of per-request logics with per-user-ad kernels on a single GPU box. This achieves over 10 billion ads per second on a single GH200 GPU.
The retrieval MoE architecture reduces inference-time FLOPs by ~300x compared to the 2024 model while delivering near ranking-level matching quality.
Fusing the entire retrieval pipeline into a single GPU kernel eliminates memory bandwidth bottlenecks, enabling over 10 billion ads per second on a single GH200 GPU.
Since 2025, Meta Andromeda has delivered a 2.4% increase in conversions on Facebook, demonstrating business impact from the co-design approach.
It uses a retrieval mixture-of-experts (MoE) architecture that reduces FLOPs by ~300x, and fuses the entire pipeline into a single GPU kernel, eliminating memory bandwidth bottlenecks. This allows over 10 billion ads per second on a single GH200 GPU.
Since 2025, Meta Andromeda has delivered a 2.4% increase in conversions on Facebook, meaning better ad relevance and more conversions for advertisers.
Previous optimizations focused on model execution only, but the new approach fuses both model and system workloads into a single GPU kernel, and co-locates per-request logics with per-user-ad kernels on a single GPU box, eliminating network and memory bottlenecks.
Summaries are generated from the original by a language model. Check the original before relying on it.
See the idea
AWS Graviton migration trades strong price-performance gains against real coordination costs. This map shows the benefit, the cost, and the conditions that decide when Arm adoption pays off.
Keep reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
One useful email. Unsubscribe anytime.