The Ad Compute Trade-Off: Allocate by Marginal Utility
Ad ranking is a compute trade-off: more resources per request can improve relevance, but GPU capacity and latency are finite. Learn how to allocate by marginal utility instead of uniformly.
MetaZhaojun ZhangProblem & solution
Ads ranking models are growing toward LLM scale faster than hardware can keep up, and the marginal impact of compute varies across requests, models, and chips. Meta built Intelligent Delivery, an ML-driven system that allocates resources across the serving stack toward the system-wide optimum. It uses a Resource Core Model with 6B sparse parameters to predict per-request marginal utility, a hierarchy of decision engines for region, hardware, and within-host allocation, and agentic AI to continuously detect and respond to system shifts. Since 2017 through the first half of 2026, Intelligent Delivery has contributed an additional ~10% incremental ranking performance on top of model and system advancements. The system shortens the optimization loop from months to weeks, with AI agents now contributing to about 80% of engineering effort on the most automated mechanism.
Ads ranking models are growing toward LLM scale faster than hardware can keep up, and the marginal impact of compute varies across requests, models, and chips. The challenge is deciding where to allocate each unit of compute to achieve globally optimal system efficiency under finite capacity and tight latency budgets.
Intelligent Delivery is an ML-driven, increasingly AI-operated adaptive optimization layer that allocates compute resources across the ads serving stack toward the system-wide optimum. It comprises three capabilities: a shared ML intelligence layer (Resource Core Model) that predicts per-request marginal utility of compute; a hierarchy of decision engines that allocate resources across regions, hardware SKUs, and within-host resources; and a self-optimizing loop using agentic AI to continuously detect system shifts and propose configuration changes.
Intelligent Delivery uses a Resource Core Model with 6B sparse and 20M dense parameters to predict per-request marginal utility of compute, outcome, and latency impact before candidate retrieval.
Allocation is decomposed into a hierarchy of decision engines: real-time traffic shifting between regions, workload-to-SKU assignment based on benchmarked efficiency, and per-request compute allocation under a tail-latency SLO.
AI agents now contribute to about 80% of engineering effort on the most automated mechanism, shortening the optimization loop from months to weeks while keeping engineers in control of final changes.
The Resource Core Model is a deep learning model with 6B sparse parameters and 20M dense parameters that runs once before candidate retrieval. It predicts each request's outcome, compute consumption, and latency impact, and is fine-tuned into specialized models that generate incremental signals for resource allocation.
Allocation is decomposed into a hierarchy of layers: a real-time system dynamically shifts traffic between regions to equalize marginal efficiency; a solver assigns workloads to hardware SKUs based on benchmarked efficiency; and within a host, per-request compute is predicted and allocated under a tail-latency SLO.
AI agents monitor the system landscape, read source code and experiment logs, and propose a handful of configuration changes. Engineers review and approve each change before it ships. In the first half of 2026, agents contributed to about 20% of optimizations, and on the most automated mechanism, about 80% of the engineering effort.
Summaries are generated from the original by a language model. Check the original before relying on it.
See the idea
Ad ranking is a compute trade-off: more resources per request can improve relevance, but GPU capacity and latency are finite. Learn how to allocate by marginal utility instead of uniformly.
Keep reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
One useful email. Unsubscribe anytime.