From User Sequences to Scaling Laws: A Multi-Stage Architecture for Meta’s Ads Ranking

Summary

Meta’s ads ranking systems process millions of candidates per second under millisecond latency budgets, creating tension between richer sequence models and efficient serving. The post presents a multi-stage architecture that moves long-history user modeling into asynchronous offline transformers, caches user-level embeddings, and combines them with fresh user and ad signals in an online ranking model. Dense tokenization places sparse features and behavioral sequences in one vocabulary, while target-aware multi-head attention learns higher-order interactions between historical behavior and the candidate ad. On real-world ads traffic, performance follows a log-linear relationship with compute across depth, enrichment, width, and sequence length, with balanced growth and diverse sequences identified as scaling levers. Together with broader model innovations, the approach contributed to a cumulative 6% conversion lift on Instagram, 3% on Facebook, and 3.5% lift in Facebook ad clicks, and serves as a core component of GEM.

Takeaways

1. 1

   The offline stage processes sequences with lengths in the thousands, generates user-level embeddings, and caches them before online ranking, allowing model capacity to increase without a proportional rise in serving resources.

2. 2

   Dense tokenization combines sparse features with behavioral sequences in a single dense vocabulary, while target-aware attention weighs past behaviors against the specific ad being scored.

3. 3

   On real-world ads traffic, performance showed a log-linear relationship with compute; balanced growth across depth, width, and sequence length, plus diverse action sequences, were identified as scaling levers.