Loading…
Topic
Resource Allocation
2 posts about Resource Allocation. Every summary links to the original.
- 2 posts
- 2 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Posts
Sandbox resourcing: burst, don't reserve
Modal Sandboxes are bursty workloads that spend most of their time using far less than peak CPU, making reservation-based pricing wasteful. The post argues that reserving capacity for each Sandbox’s peak would require machine capacity equal to the sum of all peaks, while bursting allows multiple Sandboxes to share capacity efficiently. Modal uses active-resource pricing where Sandboxes are billed for a default request when idle and pay for additional resources only when they burst. CPU can burst up to 16 cores above the request, and memory has no limit by default, though limits can be set for consistency or cost capping. Evictions are rare, with an SLO of under 0.0001% OOM rate per minute for Sandboxes under 8 GiB, and tools like exit snapshots and volumes help mitigate their impact.
Intelligent Delivery: Making Every Unit of Compute Serve Better Ads
Ads ranking models are growing toward LLM scale faster than hardware can keep up, and the marginal impact of compute varies across requests, models, and chips. Meta built Intelligent Delivery, an ML-driven system that allocates resources across the serving stack toward the system-wide optimum. It uses a Resource Core Model with 6B sparse parameters to predict per-request marginal utility, a hierarchy of decision engines for region, hardware, and within-host allocation, and agentic AI to continuously detect and respond to system shifts. Since 2017 through the first half of 2026, Intelligent Delivery has contributed an additional ~10% incremental ranking performance on top of model and system advancements. The system shortens the optimization loop from months to weeks, with AI agents now contributing to about 80% of engineering effort on the most automated mechanism.
Related topics
All topics →Weekly digest
The best engineering reads, once a week.