MetaHuayu LiAnnouncement
Meta’s Generative Ads Model (GEM): Scaling the Central Model and Compute Engine for Frontier Ads AI
Summary
Meta introduced GEM, the industry’s largest recommendation system foundation model, and its 2026 generation shifts from isolated per-domain optimizations on Facebook, Instagram, and Reels to a single central foundation model. The Kunlun architecture, co-designed with the latest GPU hardware, enables 2.5x model complexity and demonstrates LLM-like scaling laws with block-aligned sparse attention reducing sequence complexity. The LoopFM post-training technique transfers GEM’s internal representations to downstream models, doubling intelligence transfer at no additional compute cost or latency. GEM delivered 18.9% conversion improvement and 12% ad click-through rate improvement across all Meta ad surfaces and funnel stages, including 5.8% conversion uplift on Facebook, 4.1% on Instagram, and 1.6% on off-site conversion. Training efficiency doubled to 20-25% model FLOPS utilization while scaling total training FLOPs by 4x across thousands of GPUs using custom kernels and 5D parallelism.
Context
Meta operates dozens of ad surfaces including Facebook, Instagram, and Reels, each previously with independent prediction systems. Scaling a single foundation model across these heterogeneous surfaces and objectives required new architecture and training techniques.
Approach / What changed
Built the Kunlun LLM-style architecture co-designed with GPU hardware, enabling 2.5x model complexity. Used LoopFM for efficient intelligence transfer to downstream models. Created a single unified compute engine for cross-surface, cross-domain learning. Developed custom kernels and 5D parallelism to double training efficiency to 20-25% MFU while scaling FLOPs 4x.
Takeaways
GEM’s Kunlun architecture uses block-aligned sparse attention to reduce sequence complexity from quadratic to linear relative to window size, enabling longer user engagement contexts.
The LoopFM post-training technique transfers GEM’s internal representations to downstream models, doubling intelligence transfer at no additional compute cost or latency through computation sharing.
GEM achieved 18.9% conversion improvement and 12% CTR improvement across Meta’s ad surfaces, with specific uplifts of 5.8% on Facebook, 4.1% on Instagram, and 1.6% on off-site conversion for targeted users.
FAQ
How does GEM handle the latency challenge of serving a large model in real time?
GEM uses LoopFM, a post-training technique that transfers internal representations to downstream models through computation sharing, doubling intelligence transfer at no additional compute cost or latency. It also uses elastic pre-computation (Solaris) to harvest gains from the upstream model.
What architecture innovations enable GEM's scaling?
GEM uses the Kunlun architecture with block-aligned sparse attention, mixture-of-experts cross-interaction, and deep co-design with GPU hardware to enable 2.5x model complexity growth and LLM-like scaling laws.
What performance improvements did GEM deliver?
GEM delivered 18.9% conversion improvement and 12% ad click-through rate improvement across Meta's ad surfaces, including 5.8% conversion uplift on Facebook, 4.1% on Instagram, and 1.6% on off-site conversion for targeted users over the past year.
Source and freshness
- Source
- engineering.fb.com
- Published
- Added to Yomu
Summaries are generated from the original by a language model. Check the original before relying on it.
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
The best engineering reads, once a week
One useful email. Unsubscribe anytime.