# From User Sequences to Scaling Laws: A Multi-Stage Architecture for Meta’s Ads Ranking

[Meta](https://yomu.fyi/company/meta) · By Steven De Gryze, Parshva Doshi, Sean O'Byrne, Arnold Overwijk, Dinesh Ramasamy, Lee Xiong · Aug 5, 2026

## Summary

Scaling sequence models for real-time ad recommendations faces latency bottlenecks and scaling ceilings when hybrid configurations process both long user histories and sparse feature interactions online. Meta addresses these trade-offs with a multi-stage architecture that decouples heavy, asynchronous offline user modeling from lightweight, real-time online ranking. Deep upstream transformer models process sequences of thousands of actions to generate cached user embeddings, which downstream online models combine with candidate ad features using dense tokenization and target-aware multi-head attention. This framework produces predictable, LLM-style log-linear scaling curves between compute and normalized entropy without disproportionate increases in online serving costs. Implemented as part of Meta's Generative Ads Recommendation Model, the design achieved a cumulative 6% lift in Instagram conversions and a 3% lift in Facebook conversions.

## Takeaways

- Decoupling the architecture into an asynchronous offline stage for user embeddings and a low-latency online stage for ad ranking enables upstream model capacity to scale without increasing online serving costs.
- Dense tokenization and target-aware multi-head attention allow the model to learn sparse feature interactions directly from data while scoring historical actions against specific candidate ads.
- Diverse sequence composition combining multiple engagement types—such as views, clicks, and conversions—generates richer user representations than homogeneous sequences of a single action type.

**Tags:** [Architecture](https://yomu.fyi/topic/architecture), [Machine Learning](https://yomu.fyi/topic/machine-learning), [Recommendation Systems](https://yomu.fyi/topic/recommendation-systems), [Scalability](https://yomu.fyi/topic/scalability)

[Read original post](https://engineering.fb.com/2026/08/05/ml-applications/from-user-sequences-to-scaling-laws-a-multi-stage-architecture-for-metas-ads-ranking)