GenRec: Towards LLM-Native Recommendation at Netflix

Summary

Netflix developed GenRec, an LLM-backed recommendation ranker built by adapting an internal foundation model for large-scale personalization. Traditional recommendation stacks rely on thousands of hand-crafted features and specialized architectures that are costly to maintain across diverse content types. To replace manual feature pipelines, GenRec verbalizes user histories, metadata, and contexts into natural-language prompts and trains with multi-objective losses, including catalog-aware ranking and reward-weighted alignment. At inference time, the model executes in prefill-only mode on vLLM without decoding text. In large-scale online A/B testing against a mature production ranker, GenRec achieved statistically significant improvements in short-term and long-term metrics while using fewer labeled examples.

Context

Netflix's production recommendation systems depended on thousands of hand-crafted features and specialized multi-task architectures across movies, series, games, live content, and podcasts. Adding new surfaces or content types required extensive feature engineering, architectural modifications, and infrastructure investment, making new use cases costly to onboard. Meanwhile, off-the-shelf LLMs suffered from over-recommending popular items, hallucinating titles, and ignoring business rules.

Approach / What changed

GenRec uses a two-phase framework: Phase 1 adapts an open-source LLM on internal corpora for content and member behavior understanding, while Phase 2 post-trains on conversational interaction logs with catalog-aware ranking, language modeling, and reward-weighted losses. Context engineering compresses histories into high-signal natural language prompts optimized for prefix caching. Inference runs in prefill-only mode on vLLM to score catalog items directly via a catalog-aware head without generating text.

Takeaways

• GenRec runs inference in prefill-only mode on vLLM and scores items via a catalog-aware head instead of decoding text assistant messages.
• Context engineering manages token budgets by retaining high-signal events, omitting low-signal interactions, compressing repetitive behaviors, and optimizing prompts for prefix caching.
• In an online A/B test against a tuned production ranker, GenRec improved short-term and long-term metrics while using a fraction of Phase-2 labeled data and input signals.

Related reading

• Netflix · Jun 29, 2026

  GenPage: Towards End-to-End Generative Homepage Construction at Netflix

  Netflix traditionally constructs its structured two-dimensional homepage through a complex, multi-stage recommender pipeline that separates candidate generation and ranking across rows and entities. To simplify this architecture and optimize directly for whole-page user satisfaction, Netflix developed GenPage, an end-to-end generative transformer model that autoregressively builds the entire homepage from raw tokenized context. The system relies on a domain-specific tokenizer to compress engagement history, context injection for cold start, hybrid row decoding to minimize decoding steps, and reinforcement learning post-training. In online A/B testing against the production baseline, GenPage delivered statistically significant gains in core user engagement metrics while cutting end-to-end serving latency by 20 percent. Offline evaluations further showed that enriching context prompts improved recommendation quality more effectively than increasing model capacity in the current operating regime.

  Netflix Technology Blog

• Netflix · Jul 17, 2026

  In-House LLM Serving at Netflix

  Netflix established an in-house serving platform to run large language model inference directly inside existing production environments alongside traditional machine learning models. Built upon NVIDIA Triton Inference Server and vLLM, the unified architecture handles member-scale routing, candidate generation, feature fetching, and model execution over gRPC and OpenAI-compatible HTTP endpoints. The team shifted their primary engine from TensorRT-LLM to vLLM to support custom architectures, simpler debugging, and non-trivial constraint logic. Addressing production obstacles required patching Triton's frontend for guided decoding, pinning dependent library versions, rewriting logits processors in C++ for vLLM V1, and handling state machine resets during engine preemptions. The resulting platform unifies deployment flows while preserving operational stability across evolving model schemas.

  Netflix Technology Blog

• Meta · Aug 5, 2026

  From User Sequences to Scaling Laws: A Multi-Stage Architecture for Meta’s Ads Ranking

  Scaling sequence models for real-time ad recommendations faces latency bottlenecks and scaling ceilings when hybrid configurations process both long user histories and sparse feature interactions online. Meta addresses these trade-offs with a multi-stage architecture that decouples heavy, asynchronous offline user modeling from lightweight, real-time online ranking. Deep upstream transformer models process sequences of thousands of actions to generate cached user embeddings, which downstream online models combine with candidate ad features using dense tokenization and target-aware multi-head attention. This framework produces predictable, LLM-style log-linear scaling curves between compute and normalized entropy without disproportionate increases in online serving costs. Implemented as part of Meta's Generative Ads Recommendation Model, the design achieved a cumulative 6% lift in Instagram conversions and a 3% lift in Facebook conversions.

  By Steven De Gryze, Parshva Doshi, Sean O'Byrne, Arnold Overwijk, Dinesh Ramasamy, Lee Xiong

• Grab · Mar 19, 2026

  From firefighting to building: How AI agents restored our team’s core productivity

  Grab's Analytics Data Warehouse team spent roughly 40% of their engineering bandwidth answering repetitive questions, tracing data lineage, and handling basic pipeline enhancement requests across more than 15,000 tables. To eliminate these manual investigative bottlenecks, the team implemented a multi-agent AI architecture using FastAPI, LangGraph, Redis, and PostgreSQL. Incoming requests route through two dedicated pathways: an enhancement pipeline for generating code changes and an investigation pipeline for diagnosing data anomalies. Specialized agents interact with underlying engines like Trino, GitLab, and observability platforms to query data, trace transformations, and check ongoing incidents before synthesizing findings. This system automates the context-gathering process within minutes while maintaining human-in-the-loop review for merge requests and production changes.

  Sneh Agrawal