GEM Training: How Meta Doubled the Efficiency of Its LLM-Scale Ads Foundation Model

Summary

Meta's Generative Ads Recommendation Model (GEM) trains across thousands of GPUs using a hybrid architecture featuring trillions of sparse embedding parameters and billions of dense parameters. Standard training infrastructure built for large language models struggles with recommendation-specific workloads characterized by jagged user activity histories, asymmetric attention patterns, and numerical sensitivity. To overcome these bottlenecks, engineering teams introduced custom kernels like Jagged Flash Attention and Generalized Dot-Product Attention alongside MXFP8 mixed-precision execution. Pairing these kernel optimizations with topology-aware 5D parallelism doubled end-to-end training efficiency to 20–25% Model FLOPs Utilization while scaling total training FLOPs fourfold over twelve months.

Takeaways

• Padding variable-length user activity sequences can waste up to 50% of compute, which Jagged Flash Attention avoids by executing variable-length tensors directly using Tensor Memory Accelerator subtraction schemes.
• End-to-end Model FLOPs Utilization (MFU) is decoupled into single-GPU compute efficiency (Local MFU) and multi-GPU distributed retention (Scaling Ratio) to address kernel design and distributed communication as separate problems.
• GEM scales dense parameters using 2D Fully Sharded Data Parallelism combined with Expert Parallelism, while sparse embedding parameters are distributed using Fully Sharded 2D Model Parallelism.

Related reading

• Meta · Aug 5, 2026

  From User Sequences to Scaling Laws: A Multi-Stage Architecture for Meta’s Ads Ranking

  Scaling sequence models for real-time ad recommendations faces latency bottlenecks and scaling ceilings when hybrid configurations process both long user histories and sparse feature interactions online. Meta addresses these trade-offs with a multi-stage architecture that decouples heavy, asynchronous offline user modeling from lightweight, real-time online ranking. Deep upstream transformer models process sequences of thousands of actions to generate cached user embeddings, which downstream online models combine with candidate ad features using dense tokenization and target-aware multi-head attention. This framework produces predictable, LLM-style log-linear scaling curves between compute and normalized entropy without disproportionate increases in online serving costs. Implemented as part of Meta's Generative Ads Recommendation Model, the design achieved a cumulative 6% lift in Instagram conversions and a 3% lift in Facebook conversions.

  By Steven De Gryze, Parshva Doshi, Sean O'Byrne, Arnold Overwijk, Dinesh Ramasamy, Lee Xiong

• Meta · Jul 15, 2026

  Exploring Hierarchical Interest Representation For Meta Ads Deep Funnel Optimization

  Meta developed Hierarchical Interest Representation as an upstream representation layer to optimize deep funnel ad ranking across its platforms. The architecture organizes interactions among users, advertisers, and products into a heterogeneous, time-decayed graph and projects sparse raw edges into multi-level super-graphs of latent interest primitives. To encode these structures, a transformer-based hierarchical encoder incorporates multimodal product semantics processed via language models alongside topology-aware attention biases. Computation scales across billions of entities by pairing deep hash ID embeddings with FlexAttention to evaluate graph biases on the fly without materializing full matrices. Self-supervised cross-view distillation trains student representations against broader teacher graph views to yield universal embeddings and interest tokens.

  By Yuhui Ouyang, Di Wang, Sreedal Menon, Jie Tian

• Meta · Aug 24, 2026

  MetaRoCE: A New RDMA Transport Built for AI-Scale Ethernet

  Scaling frontier AI training and inference across massive GPU clusters places severe demands on network fabrics, yet conventional RoCE relies on Priority Flow Control to enforce in-order packet delivery. Meta introduced MetaRoCE, a clean-sheet RDMA transport protocol engineered specifically for lossy, multiplane commodity Ethernet without requiring PFC. By shifting intelligence to endpoint NICs, the protocol sprays packets across fine-grained logical paths, writing out-of-order data directly into destination memory without reorder buffers or head-of-line blocking. Congestion is managed via per-path ECN windows paired with receiver-driven rate hints returned in acknowledgments. In validation on a 64-node AMD GPU cluster running collective operations, MetaRoCE outperformed RoCEv2, sustaining roughly 86% throughput under 1% packet loss and achieving linear scaling across multiplane fabrics.

  By Arvind Srinivasan, Neil Spring, Omar Baldonado, Rajiv Krishnamurthy

• Meta · Aug 24, 2026

  MTIA 300: Meta’s First Training Chip with Built-in NICs and Communication-Offloading Engines

  Training recommendation models poses distinct network bottlenecks because massive embedding tables trigger frequent collective operations that compete with computation on standard GPUs. Meta designed the MTIA 300 accelerator to overcome this by embedding two network chiplets containing twelve 800 Gbps RDMA NICs directly inside the chip package. The architecture offloads communication from its compute grid to sixteen dedicated message engines equipped with RISC-V cores and near-memory compute reduction blocks. Working alongside the co-designed HCCL library, the chip autonomously compiles and executes collective communication subgraphs without host CPU intervention. On a 150-billion-parameter production recommendation model, MTIA 300 delivered communication speeds 3.9 times faster than an equivalent GPU cluster while keeping concurrent compute degradation under 0.5 percent.

  By Rajiv Krishnamurthy, Wes Bland