The Ad Compute Trade-Off: Allocate by Marginal Utility
Ad ranking is a compute trade-off: more resources per request can improve relevance, but GPU capacity and latency are finite. Learn how to allocate by marginal utility instead of uniformly.
AWSNishita KheurProblem & solution
Zillow sends over 100 million daily notifications using a centralized, configuration-driven orchestration layer called Campaign Orchestrator. It runs each notification as a short-lived workflow on AWS Step Functions Express Workflows, chosen for being serverless and lower cost than alternatives. A router applies campaign controls like queue polling, batching, and circuit breakers before starting each workflow. The workflows call reusable, schema-driven content modules from the Universal Content Generator library, with JSONata expressions validated via the TestState API. The new architecture cut build effort from 32 to 9 weeks and lowered ongoing maintenance from 8 to 2 hours per week.
Zillow needed to scale its notification system from 14 million to over 100 million daily workflows while reducing duplicated engineering work and operational overhead. The previous system used separate microservices for each campaign, leading to inconsistent behavior and high maintenance costs.
Zillow built a configuration-driven orchestration layer called Campaign Orchestrator that runs each notification as a short-lived workflow on AWS Step Functions Express Workflows. A router applies campaign controls like queue polling, batching, and circuit breakers before starting each workflow. The workflows call reusable, schema-driven content modules from the Universal Content Generator library. JSONata expressions are validated using the Step Functions TestState API.
Zillow replaced separate microservices for each campaign with a centralized, configuration-driven orchestration layer called Campaign Orchestrator.
Step Functions Express Workflows were chosen over Temporal and Flowable because they are serverless, lower cost, and require no infrastructure management.
Each campaign gets its own queue with dedicated polling threads and circuit breakers to prevent failures from affecting other campaigns.
Express Workflows have higher default start quotas, are optimized for short-running workflows, and provide at-least-once execution which is sufficient with idempotent modules. Standard Workflows have lower quotas and exactly-once execution which is stronger than needed.
They keep shared JSONata expressions in dedicated files and require unit tests that run through the Step Functions TestState API, validating expressions against the same runtime used by Step Functions. Failing tests block deployment.
It is a library of reusable, schema-driven content modules that the Campaign Orchestrator workflows call. It provides consistent content generation across campaigns and allows AI-assisted tooling to work with a declarative format.
Summaries are generated from the original by a language model. Check the original before relying on it.
See the idea
Ad ranking is a compute trade-off: more resources per request can improve relevance, but GPU capacity and latency are finite. Learn how to allocate by marginal utility instead of uniformly.
Keep reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
One useful email. Unsubscribe anytime.