The Ad Compute Trade-Off: Allocate by Marginal Utility
Ad ranking is a compute trade-off: more resources per request can improve relevance, but GPU capacity and latency are finite. Learn how to allocate by marginal utility instead of uniformly.
Loading…
Topic
3 posts about Ads. Every summary links to the original.
See the idea
Ad ranking is a compute trade-off: more resources per request can improve relevance, but GPU capacity and latency are finite. Learn how to allocate by marginal utility instead of uniformly.
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Meta’s ads ranking system comprises many distinct models, each with unique architecture, training data, and serving constraints. Propagating proven ML techniques across this fleet is challenging because techniques rarely transfer directly, and speed matters as innovations have a shelf life. To scale, Meta first standardized models using reusable templates, reducing per-model engineering time by 92% and shipping 5,000+ technique-model combinations. Now, AI agents handle over 60% of technique propagation by automating discovery, generalization, and adoption, achieving 3x engineering productivity, a median 4x reduction in training compute, and an estimated $45M+ in compute savings. The agents use LLM-based reasoning and a shared knowledge base to make judgment calls, with evaluation metrics including precision and recall for discovery, normalized entropy for generalization, and human intervention rate for adoption, as Meta aims to reduce propagation time from over a year to three months.
Meta’s ads ranking ecosystem, processing billions of daily requests, faced a bottleneck where propagating new techniques across a large and fragmented model fleet became harder than the innovation itself. To solve this, they introduced Standard Model Templates (SMT), which standardize model design into a small set of reusable building blocks covering architecture, features, training, and serving. This shifts iteration from per-model to template-level, reducing complexity and enabling multi-model optimization that evaluates techniques across representative models. Over two years, SMT reduced per-model engineering time by 92%, shipped over 5000 technique-model combinations, improved offline model quality by 0.63% at neutral serving capacity, and delivered a 0.86% cumulative online lift covering 70% of models. The approach also provides a structured foundation for agentic ML systems, where agents explore and optimize within bounded template spaces, making automation more effective.
Ads ranking models are growing toward LLM scale faster than hardware can keep up, and the marginal impact of compute varies across requests, models, and chips. Meta built Intelligent Delivery, an ML-driven system that allocates resources across the serving stack toward the system-wide optimum. It uses a Resource Core Model with 6B sparse parameters to predict per-request marginal utility, a hierarchy of decision engines for region, hardware, and within-host allocation, and agentic AI to continuously detect and respond to system shifts. Since 2017 through the first half of 2026, Intelligent Delivery has contributed an additional ~10% incremental ranking performance on top of model and system advancements. The system shortens the optimization loop from months to weeks, with AI agents now contributing to about 80% of engineering effort on the most automated mechanism.
The best engineering reads, once a week.