---
title: "Recommendation Systems"
description: "55 posts about Recommendation Systems, summarised, each linking to the original."
---

# Recommendation Systems
> 55 posts about Recommendation Systems, summarised, each linking to the original.

## Articles

### [What is Vector Search?](https://yomu.fyi/post/what-is-vector-search.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Databricks Staff
- Published: Jun 23, 2026
- Type: Explainer

Vector search retrieves results by comparing embeddings that represent meaning across text, images, audio, and other content rather than matching exact words. A model creates embeddings, an index stores them for fast similarity search, and a query embedding is matched against the index using nearest-neighbor methods. Exhaustive k-nearest neighbor search can become too slow at millions of items, so production systems commonly use approximate nearest neighbor search, trading some precision for speed. The guide positions vector search behind semantic search, RAG, recommendations, and multimodal or cross-language retrieval, while hybrid search combines dense and sparse vectors, keyword results, metadata filtering, and reranking to improve reliability. Quality depends on embeddings, filters, index freshness, and infrastructure, with vector search requiring more memory and compute; Databricks AI Search is presented as a managed service supporting these capabilities and Unity Catalog governance.


### [Making User-Sequence Data More Cost-Efficient, Faster, and Easier to Use](https://yomu.fyi/post/making-user-sequence-data-more-cost-efficient-faster-and-easier-to-use.md)
- Company: [Pinterest](https://yomu.fyi/company/pinterest.md)
- Author: Pinterest Engineering
- Published: May 21, 2026
- Type: Problem & solution

User sequences serve as core primitives powering machine learning models across ranking, retrieval, and recommendation surfaces at Pinterest. Managing these sequences previously caused high infrastructure costs, operational fragility, and data drift between offline training pipelines and online serving systems. In response, engineering teams redesigned the platform around a single definition executed across multiple runtimes, including real-time streaming, batch backfills, and online inference. The updated architecture pairs a unified enrichment engine with columnar time-partitioned storage and standardized validation tiers. Consequently, the redesign achieved lower storage and network costs, faster signal onboarding via configuration, and improved engagement metrics across major recommendation surfaces.


### [Enhancing Ad Relevance: Integrating Real-Time Context into Sequential Recommender Models](https://yomu.fyi/post/enhancing-ad-relevance-integrating-real-time-context-into-sequential-r.md)
- Company: [Pinterest](https://yomu.fyi/company/pinterest.md)
- Author: Pinterest Engineering
- Published: May 8, 2026
- Type: Problem & solution

Pinterest's sequential ads candidate generator inferred user embeddings offline from historical offsite conversions, which lacked real-time browsing context. Consequently, less than 1% of impressions on the Related Pins surface were attributed to this generator because its retrieved candidates struggled in downstream ranking. To resolve this limitation, engineers developed the Contextual Sequential Two Tower Model, integrating a context layer directly into the query tower alongside user demographic features. Offline training utilized synthetic context derived from positive conversion labels paired with high dropout rates to preserve sequence reliance, while serving split into offline Transformer inference and online context computation. Offline evaluations showed a 3x to 10x increase in Recall@K, and production deployment lifted median candidate relevance by 275% to 300% and overall ROAS by approximately 0.7%.


### [From Clicks to Conversions: Architecting Shopping Conversion Candidate Generation at Pinterest](https://yomu.fyi/post/from-clicks-to-conversions-architecting-shopping-conversion-candidate.md)
- Company: [Pinterest](https://yomu.fyi/company/pinterest.md)
- Author: Pinterest Engineering
- Published: Apr 27, 2026
- Type: Problem & solution

Pinterest built a dedicated shopping candidate generation model to optimize for lower-funnel purchase conversions rather than relying strictly on engagement retrieval models. Because offsite conversion events are sparse and delayed, engineers trained a single multi-surface two-tower model using conversion data supplemented by duration-weighted click engagement and unengaged ad impressions as hard negatives. The two-tower retrieval architecture incorporates a parallel DCN v2 and MLP layer structure that allows simultaneous learning from the same inputs without information bottlenecks. Later revisions transitioned the system from a multi-head loss structure to a unified single-head multi-task architecture paired with an advertiser-level objective. Deployed across more than 600 million monthly active users, the system increased shopping conversion volume by 2.3% and boosted return on ad spend by 3.1% for United States shopping campaigns.


### [How We Built a Smarter Pickup Experience for Gated Communities](https://yomu.fyi/post/how-we-built-a-smarter-pickup-experience-for-gated-communities.md)
- Company: [Lyft](https://yomu.fyi/company/lyft.md)
- Author: winnieyan
- Published: Apr 23, 2026
- Type: Problem & solution

Pickups in gated communities account for up to 30% of Lyft rides in certain markets, yet locked gates and lack of advance instructions historically resulted in higher cancellation rates and extended wait times. To resolve these friction points, Lyft's Mapping team developed an end-to-end mapping and routing framework. The team built an algorithm to generate gate area boundaries, enabling the app to detect gated communities and offer riders explicit choices between waiting inside or meeting outside the gate. Routing services incorporate the gate as an intermediate waypoint to direct drivers along the proper approach path while triggering timely gate access instructions. Following deployment, rides with shared gate instructions exhibited lower cancellation rates from both parties, shorter walking distances to pickup spots, and fewer extended driver wait times.


### [Personalization Strategies for Media Companies](https://yomu.fyi/post/personalization-strategies-for-media-companies.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Databricks Staff
- Published: Apr 22, 2026
- Type: Explainer

Media companies are expected to tailor content and messaging across channels, with personalization programs typically targeting lower churn, deeper sessions, and higher subscription revenue. The proposed foundation starts with an audit of CRM, email, warehouse, and customer data platform sources, followed by required-field definitions, ownership mapping, consent tagging, and ingestion-quality checks. Teams then map segment behaviors and conversion or retention paths, including negative signals, before selecting a CDP with real-time ingestion, identity resolution, unified profiles, segmentation, activation, and governance. Identity rules should connect anonymous and authenticated activity across web, mobile, and connected TV, while event-level streaming delivers messages through email, push, in-app, and advertising channels with minimal latency. The roadmap recommends an MVP comprising unified ingestion, a homepage recommendation surface, and triggered re-engagement email, with performance, churn, revenue lift, and model drift monitored through iteration.


### [Ecom-RLVE: Adaptive Verifiable Environments for E-Commerce Conversational Agents](https://yomu.fyi/post/ecom-rlve-adaptive-verifiable-environments-for-e-commerce-conversation.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Rahul Bajaj, Jaya Nupur, Anuj Garg, ben burtenshaw
- Published: Apr 16, 2026
- Type: Explainer

Ecom-RLVE extends the RLVE framework from single-turn reasoning puzzles to multi-turn, tool-augmented e-commerce conversations. EcomRLVE-GYM supplies eight environments spanning product discovery, substitution, cart building, returns, order tracking, policy QA, bundle planning, and multi-intent journeys, with procedural problem generation, a 12-axis difficulty curriculum, and algorithmically verifiable rewards. Each episode requires tool use and world-state changes; rewards combine task completion, efficiency, and a penalty for recommending product IDs not retrieved, while malformed outputs and illegal calls fail immediately. Adaptive scheduling advances each environment independently when its success rate is reliable. An initial study trained Qwen 3 8B with DAPO for 300 steps on cart building and reported progressively increasing difficulty reached, consistent with a steady learning signal.


### [Building a Magic Mirror: AI retail experiences with Remix](https://yomu.fyi/post/building-a-magic-mirror-ai-retail-experiences-with-remix.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Mar 19, 2026
- Type: Tutorial

Shopify’s “magic mirror” is an AI-powered retail installation that uses a display and hidden webcam to deliver personalized messages, animations, and product interactions in under a minute. A Remix route handles the browser and server boundary: a button-triggered POST captures an image, sends it with a curated prompt to a vision-capable language model, and renders a compliment plus a makeup recommendation; server-sent events optionally let a remote device trigger or reset the experience. The prompt combines persona and safety constraints, a plain-text product and inventory table, and a required JSON schema, while strict product-ID lookup provides graceful degradation. The interface uses explicit states, delayed capture, timed response and reset screens, and animated transitions. The production setup keeps brand configuration, shared components, actions, and SSE infrastructure reusable across mirror modes.


### [The generative recommender behind Shopify's commerce engine](https://yomu.fyi/post/the-generative-recommender-behind-shopify-s-commerce-engine.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Feb 25, 2026
- Type: Explainer

Shopify describes a foundational generative recommender that predicts next products, and in some contexts ads, from buyer event sequences spanning searches, views, add-to-carts, favorites, and purchases. The system uses an autoregressive model with causal masking, time-aware attention combining RoPE-inspired rotary encoding with relative attention bias, and sampled softmax with shared and positive-aware hard negatives. Built on HSTU, it combines richer activity data, ensemble-focused training for incremental recall, and optimized CUDA kernels; the full training pipeline reached up to 7.3x the speed of a baseline implementation. In an online test of an August model version, relative shop orders rose 0.94%, high-quality click-through rate rose 5%, conversion rate rose 0.71%, and final served product recall at 2 rose 4.8%.


### [Building a Movie Recommendation Engine with Hugging Face and Voyage AI](https://yomu.fyi/post/building-a-movie-recommendation-engine-with-hugging-face-and-voyage-ai.md)
- Company: [MongoDB](https://yomu.fyi/company/mongodb.md)
- Author: Arek Borucki
- Published: Feb 17, 2026
- Type: Tutorial

Traditional movie search mechanisms depend on coarse filters such as genre, actor, or title, which fail to capture emotional context and narrative nuances. This guide demonstrates building a mood-based semantic recommendation system by combining the voyage-4-nano open-weights embedding model, Hugging Face datasets, and MongoDB Atlas Vector Search. The architecture uses Sentence Transformers to generate embeddings and exposes an endpoint through FastAPI to match user moods against movie plots. Truncating the Matryoshka embeddings from 2048 to 1024 dimensions balances semantic retrieval quality, storage footprint, and query latency. Testing shows that while abstract emotional queries return moderate similarity scores around 0.62 to 0.67, concrete descriptive queries exceed 0.75 without requiring exact keyword matches.


### [User foundation models for Grab](https://yomu.fyi/post/user-foundation-models-for-grab.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Abhinav Rai
- Published: Sep 26, 2025
- Type: Explainer

Grab describes a foundation model that learns from both tabular data and sequential clickstream interactions to represent users, merchants, and drivers across its superapp. Traditional recommender systems depended on manually engineered, task-specific features, which required substantial effort and did not capture evolving behavior holistically. The custom architecture addresses heterogeneous modalities including text, numerical values, IDs, images, and location data, combining long-term profiles with immediate intent. Grab pre-trains the model on a diverse corpus, then either extracts embeddings for downstream recommendation and prediction tasks or fine-tunes it to predict directly. The resulting reusable representations are reported to support applications such as ad optimisation, dual app prediction, fraud detection, and churn probability, while future work targets unified data, improved architecture, and greater training and inference efficiency.


### [Engineering stories behind the Medium Daily Digest Algorithm: Part 2](https://yomu.fyi/post/engineering-stories-behind-the-medium-daily-digest-algorithm-part-2.md)
- Company: [Medium](https://yomu.fyi/company/medium.md)
- Author: Raphael Montaud
- Published: Aug 28, 2025
- Type: Problem & solution

Medium’s Daily Digest filtering had become costly and difficult to maintain after a single Bloom Filter instance accumulated billions of items for read, muted, and presentation rules. The system could face up to 15,000 filter questions per feed and more than 15 million per second across thousands of feeds, while Bloom Filters could not delete or enumerate entries and introduced derived-data drift. The team replaced the muted-writer filter with a user-based database query that fetched all muted writers for the current user and cross-referenced them with the story shortlist, then split read filtering between early user-based queries and later pairwise lookups. The resulting implementation removed Bloom Filters entirely, was reported as ten times cheaper in their situation, made costs independent of shortlist size, supported reading-history clearing, and was easier to control and debug.


### [Engineering stories behind the Medium Daily Digest Algorithm: Part 1](https://yomu.fyi/post/engineering-stories-behind-the-medium-daily-digest-algorithm-part-1.md)
- Company: [Medium](https://yomu.fyi/company/medium.md)
- Author: Raphael Montaud
- Published: Aug 26, 2025
- Type: Problem & solution

Medium’s Daily Digest recommendations were less relevant than the homepage despite identical story sourcing, ranking model, and features, because digest filtering operated with less email-reporting information and was too aggressive. The “Sent in Opened Digest” filter excluded all 15 stories after an email-open signal, even when a reader saw only the first five; Apple Mail Privacy Protection also caused servers to preload messages, making many open events unreliable. Medium removed that filter and reduced the backoff from seven to four days, accepting more repetition while relying on other filtering rules and organic recommendation changes to diversify digests. An A/B test found that experiment users were 10% more likely to convert to paying members and read 7% more on Medium. The post says these results led to more filter investigations, including Bloom Filters, “Hard vs Soft Filtering,” and cross-digest diversification, while recommending audits of products that depend on email-open tracking.


### [Engineering stories behind the Medium Daily Digest Algorithm: Part 4](https://yomu.fyi/post/engineering-stories-behind-the-medium-daily-digest-algorithm-part-4.md)
- Company: [Medium](https://yomu.fyi/company/medium.md)
- Author: Raphael Montaud
- Published: Aug 25, 2025
- Type: Problem & solution

Medium’s Daily Digest team investigated complaints that recommendations had become repetitive, especially for power users, after earlier filtering changes. Data showed some stories were sent up to eight times to one user, while affected users saw digest posts repeated twice on average; the team also had to account for Apple Mail Privacy Protection, which prevents reliable open tracking. They therefore tracked digest email IDs when readers clicked a story, marking emails with at least one click as “definitely opened,” and added a recommendation-stack filter that counts prior occurrences in those digests. Testing supported a threshold of one occurrence: complaining users would lose nearly five stories per digest, whereas free users would lose fewer than one. An A/B test produced a statistically significant increase in reading time, with a larger rise among members; the team reported no further complaints and suggested soft filtering and email-position rules as possible refinements.


### [Leveraging multimodal LLMs for Shopify’s global catalogue: Recap of expo talk at ICLR 2025](https://yomu.fyi/post/leveraging-multimodal-llms-for-shopify-s-global-catalogue-recap-of-exp.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Jul 16, 2025
- Type: Explainer

Shopify’s Global Catalogue addresses fragmented, multilingual, multimodal product data that makes semantic search, faceting, matching, and AI-driven discovery difficult across millions of merchants. The system combines product data, understanding, matching, and reconciliation layers to standardize metadata, identify duplicate listings, and create canonical records with Universal Product IDs. Shopify fine-tunes vision large language models for multiple tasks per catalogue entity, while streaming infrastructure processes more than 10 million product updates daily and serves 40 million multimodal LLM-powered inferences. The resulting data supports merchant-admin suggestions, search, recommendations, embeddings, and conversational commerce, although scalability, accuracy, latency, model consolidation, graph reasoning, and continuous retraining remain active challenges.


### [Augmented commerce: Machine learning at Shopify](https://yomu.fyi/post/augmented-commerce-machine-learning-at-shopify.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Jul 4, 2025
- Type: Explainer

Shopify frames commerce as a complex, continuous optimization problem in which merchants have different definitions of success and need support across the business lifecycle. Its machine-learning work uses a historical dataset covering millions of businesses to optimize product metadata, transaction fraud assessment, merchant assistance, product discovery, capital decisions, behavioral modeling, and search query rewriting. The systems include fine-tuned Qwen multimodal models, LLaMa models and MCPs for Sidekick, Nomic embeddings for billions of products, a tabular transformer for forecasting merchant GMV, and experiments with HSTU sequence models. To support this work, Shopify partners with GCP, Nebius, CentML, and Toloka, while providing access to GPUs, data services, frontier models, and internal tooling; the post also presents these efforts as part of its hiring pitch.


### [Evolution of Product Classification at Shopify: From Categories to Comprehensive Product Understanding](https://yomu.fyi/post/evolution-of-product-classification-at-shopify-from-categories-to-comp.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: May 8, 2025
- Type: Explainer

Shopify’s product classification system evolved from a 2018 logistic-regression model using TF-IDF into a product-understanding platform designed to capture categories, attributes, characteristics, metadata, and content-safety signals. A 2020 multimodal image-and-text approach improved classification, but growing product complexity led Shopify to combine Vision Language Models with its Standard Product Taxonomy, spanning more than 10,000 categories and over 1,000 attributes across more than 26 business verticals. The production pipeline uses two-stage category and attribute prediction, with category context shaping the second call, and runs on Kubernetes with NVIDIA GPUs; FP8 quantization, Nvidia Dynamo in-flight batching, and KV caching improve inference efficiency. A multi-LLM annotation system independently evaluates products and uses arbitration when models disagree, helping build training data for consistent predictions. The resulting system processes over 30 million predictions daily, delivers an 85% predicted-category acceptance rate, and doubles hierarchical precision and recall versus the earlier neural network approach.


### [eBay Uses Agentic AI to Supercharge Personalized Ecommerce](https://yomu.fyi/post/ebay-uses-agentic-ai-to-supercharge-personalized-ecommerce.md)
- Company: [Ebay](https://yomu.fyi/company/ebay.md)
- Author: eBay News Team
- Published: May 6, 2025
- Type: Announcement

eBay announces a new AI shopping agent designed to provide personalized product recommendations and guidance throughout the marketplace and buying journey. It uses shopping preferences, user requests, and predictive inline messaging to support activities such as finding gifts or assembling a Spring Break wardrobe. The agent extends eBay’s shift from simple chatbots toward conversational and interactive systems intended to understand, interact with, and anticipate customer needs, combining agentic AI with eBay’s infrastructure, customer insights, and product knowledge. eBay says its Responsible AI team collaborates on AI-powered features to align them with safety, fairness, transparency, and accountability, while the new experience starts with a small percentage of U.S. customers and will expand on a rolling basis.


### [Tinder’s migration to Elasticsearch 8](https://yomu.fyi/post/tinder-s-migration-to-elasticsearch-8.md)
- Company: [Tinder](https://yomu.fyi/company/tinder.md)
- Author: Tinder
- Published: Apr 1, 2025
- Type: Problem & solution

Tinder modernized its recommendation system by migrating its core Elasticsearch infrastructure from an end-of-life Elasticsearch 6 deployment to Elasticsearch 8 on Kubernetes. The legacy setup relied on manual EC2 provisioning and custom scripts, preventing the engineering team from utilizing modern vector search capabilities while incurring substantial operational overhead. Tinder implemented Elastic Cloud on Kubernetes integrated with its in-house Infrastructure as Code framework, Scaffold, while refactoring custom Java ranking plugins to handle Elasticsearch 8 breaking changes like prohibited negative scoring. The migration was validated through offline benchmarking, online verification, and a three-wave A/B testing rollout. This transition completed with zero outages and under 0.2% data discrepancy, reducing p99 search latency by 12% to 56% and cutting annual infrastructure costs by over one million dollars.


### [Developing a modern architecture for energy utilities with embedded finance](https://yomu.fyi/post/developing-a-modern-architecture-for-energy-utilities-with-embedded-fi.md)
- Company: [Stripe](https://yomu.fyi/company/stripe.md)
- Author: Rajan Patel
- Published: Dec 17, 2024
- Type: Explainer

Energy utilities can use embedded finance to create marketplace-like experiences for tariffs, solar installations, smart-home devices, storage, and related services while capturing more energy-related revenue. The proposed foundation combines Stripe Online Payment Flows, Billing, Payments, Connect, and Data Pipeline with AWS serverless services, including Lambda, EventBridge, DynamoDB, Redshift, S3, and QuickSight. A tariff enrollment or payment can trigger webhooks and real-time event processing, while transaction and customer data support dashboards, usage insights, personalized recommendations, subscriptions, and broader partner offerings. The architecture also describes PCI Level 1 security, encryption, Radar, Strong Customer Authentication, AWS IAM, KMS, Shield, WAF, Config, Security Hub, and CloudWatch for compliance and monitoring. It concludes that this integrated payment, analytics, and security foundation can help utilities improve customer engagement, support sustainable programs, and develop new revenue streams.


[Newer posts](https://yomu.fyi/topic/recommendation-systems.md) · [Older posts](https://yomu.fyi/topic/recommendation-systems/page/3.md)