---
title: "How Generative Recommenders Are Redefining RecSys at Scale"
description: "Generative recommenders (GRs) recast recommendation as next-item or next-action sequence prediction conditioned on user histories, replacing traditional embedding-similarity objectives and potentially unifying retrieval and ranking. The post presents HSTU, which uses timestamp-ordered interleaved items and actions, SiLU-based attention weighting, relative attention bias, and gating, alongside Semantic IDs, which derive a smaller vocabulary through hierarchical clustering of item embeddings. For production, recsys-examples combines TorchRec, DynamicEmb, Megatron-Core, recommender-specific KV and decoding components, while nv-embedding-cache provides tiered GPU, CPU, and remote embedding storage. On an H100, the Semantic ID path was reported as 1.85x higher in online throughput and 46% lower median latency than SGLang under the stated long-context, beam-search workload, and DLRM v3 reached 99,997 queries/sec."
---

# How Generative Recommenders Are Redefining RecSys at Scale

[NVIDIA](https://yomu.fyi/company/nvidia-developer-blog) · Elizabeth Goodman · Aug 20, 2026

**Type:** Explainer

## Summary

Generative recommenders (GRs) recast recommendation as next-item or next-action sequence prediction conditioned on user histories, replacing traditional embedding-similarity objectives and potentially unifying retrieval and ranking. The post presents HSTU, which uses timestamp-ordered interleaved items and actions, SiLU-based attention weighting, relative attention bias, and gating, alongside Semantic IDs, which derive a smaller vocabulary through hierarchical clustering of item embeddings. For production, recsys-examples combines TorchRec, DynamicEmb, Megatron-Core, recommender-specific KV and decoding components, while nv-embedding-cache provides tiered GPU, CPU, and remote embedding storage. On an H100, the Semantic ID path was reported as 1.85x higher in online throughput and 46% lower median latency than SGLang under the stated long-context, beam-search workload, and DLRM v3 reached 99,997 queries/sec.

## Context

Traditional recommender systems face terabyte- or petabyte-scale user-history data, sparse long-tail interactions, cold-start users and items, and strict latency requirements. Static embedding tables can exceed a GPU's HBM capacity, while production serving must retrieve and rank thousands of candidates within milliseconds.

## Approach / What changed

The described stack uses generative recommendation with HSTU and Semantic IDs. recsys-examples provides TorchRec and DynamicEmb for dynamic, sharded embedding tables, Megatron-Core parallelism for dense layers, and specialized decoding components. nv-embedding-cache adds hierarchical GPU, CPU, and remote storage for embedding lookups, with caching, sharding, and PyTorch-compatible modules.

## Takeaways

- HSTU replaces softmax attention normalization with SiLU-based weighting, adds relative attention bias, and applies elementwise gating before output projection to support efficient long-sequence recommendation.
- Semantic IDs create a smaller vocabulary through hierarchical clustering of item embeddings, enabling autoregressive generation, beam search, and ranking from output logits.
- In the stated H100 benchmark, Semantic ID-GR serving achieved about 19.7 requests per second versus 10.7 for SGLang, with about 198 ms versus 370 ms median latency.

**Tags:** [Caching](https://yomu.fyi/topic/caching), [Machine Learning](https://yomu.fyi/topic/machine-learning), [Performance](https://yomu.fyi/topic/performance), [Recommendation Systems](https://yomu.fyi/topic/recommendation-systems)

- Source: [NVIDIA](https://developer.nvidia.com/blog/how-generative-recommenders-are-redefining-recsys-at-scale)
- Source URL: https://developer.nvidia.com/blog/how-generative-recommenders-are-redefining-recsys-at-scale
- Ingested by Yomu: 2026-08-29T00:00:59.661Z

[Read original post](https://developer.nvidia.com/blog/how-generative-recommenders-are-redefining-recsys-at-scale)