Building real-time product search on Databricks

Summary

Real-time product search for an online marketplace must retrieve, filter, rank, and respond in milliseconds while balancing latency, relevance, revenue, and click-through rate. The Databricks architecture divides the pipeline into ingestion, retrieval, and refinement, using Lakeflow pipelines and AI Functions for source processing, AI Search for embeddings and hybrid retrieval, and Lakebase for sub-10ms operational signals. It recommends validating p95/p99 latency, throughput, retrieval quality, and engagement metrics, with MLflow supporting systematic evaluation and caching helping meet strict latency requirements. As an example, FOX Sports used AI Search for thousands of QPS and achieved a 2x improvement in query success rate, combining streaming ingestion, two-phase retrieval, model serving, caching, and trending searches.

Takeaways

1. 1

   The search pipeline has three functional segments: ingestion prepares and indexes product data, retrieval generates candidates through full-text, semantic, or hybrid search, and refinement applies intent, ranking, personalization, and business rules.

2. 2

   Lakebase provides session state, inventory, pricing, and user preferences with sub-10ms latency, while managed change-data-capture syncs operational data to Delta without custom pipelines.

3. 3

   FOX Sports reported handling thousands of QPS with a 2x improvement in query success rate by combining streaming ingestion, two-phase retrieval, model serving, caching, and trending searches.