The generative recommender behind Shopify's commerce engine

Summary

Shopify describes a foundational generative recommender that predicts next products, and in some contexts ads, from buyer event sequences spanning searches, views, add-to-carts, favorites, and purchases. The system uses an autoregressive model with causal masking, time-aware attention combining RoPE-inspired rotary encoding with relative attention bias, and sampled softmax with shared and positive-aware hard negatives. Built on HSTU, it combines richer activity data, ensemble-focused training for incremental recall, and optimized CUDA kernels; the full training pipeline reached up to 7.3x the speed of a baseline implementation. In an online test of an August model version, relative shop orders rose 0.94%, high-quality click-through rate rose 5%, conversion rate rose 0.71%, and final served product recall at 2 rose 4.8%.

Takeaways

1. 1

   Time is encoded directly through a RoPE-inspired rotary encoding and relative attention bias, allowing the model to represent absolute time, time gaps, recency, and the current session timestamp.

2. 2

   Shared negatives expand the sampled softmax pool across a batch, while positive-aware hard negatives avoid treating potential true positives as negatives and focus learning on informative near misses.

3. 3

   An August online model test reported relative lifts of 0.94% in shop orders, 5% in high-quality click-through rate, 0.71% in conversion rate, and 4.8% in final served product recall at 2.