AI Search: Speed vs. Relevance Trade-off
AI Search balances retrieval speed and result quality by choosing between dense embeddings, sparse lexical search, and hybrid approaches. Understand which trade-off fits your workload.
Loading…
Topic
AI Search
7 posts about AI Search. Every summary links to the original.
- 7 posts
- 4 companies
- 1 explainer
See the idea
AI Search explainers
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Posts
Cloudflare and OpenAI: making AI discovery better for everyone
Cloudflare and OpenAI launched a research pilot in July 2026 to help ChatGPT users discover content more efficiently using network signals about page existence changes and potential harm. The pilot ran on a cohort of about one percent of Cloudflare’s network and has enabled OpenAI to index hundreds of millions of new web pages that are useful but sparsely linked. Cloudflare provides these discovery signals to OpenAI’s crawler while OpenAI independently evaluates and ranks pages by relevance and helpfulness for ChatGPT Search. The collaboration also tests signals that flag harmful pages like fake login forms so crawlers avoid them without penalizing the legitimate domain. The next phase will refine page selection test faster change signals and measure business outcomes for site owners whose content is discovered.
Real-Time Retail Intelligence: Building E-Commerce Recommendations with Lakebase and AI Search on Databricks
The post presents a reference architecture for building a production-grade e-commerce recommendation system on Databricks, based on a real-world implementation for a leading fashion e-commerce platform in Asia serving over 1 million monthly active users across 100,000+ SKUs. The system ingests approximately 1,000 events per second via Lakeflow Connect’s Zerobus into Unity Catalog Delta tables within a medallion architecture. Serving uses two paths: Path A for pre-computed batch recommendations via key-value lookup from Lakebase online tables, and Path B for real-time session-aware scoring using AI Search and a LightGBM model in a 3-stage funnel. The architecture unifies ingestion, feature management, model training, vector retrieval, and serving on a single governed platform, with cold-start handling for new users and products. Expected outcomes include low 2-digit millisecond responses, daily-fresh recommendations, and unified governance via Unity Catalog.
Pay Per Use: when AI uses your work, you should get paid
AI answer engines can read a publisher’s page and deliver a summary without the reader ever visiting the site, costing the publisher the visit and its associated revenue. Most publishers cannot negotiate individual licensing deals, and no AI company can contract with millions of sites. Pay Per Use, now in beta, is Cloudflare’s system that lets a buyer define a specific use of content and a price, and lets each publisher choose whether to accept that offer. The buyer reports each downstream use through a single Cloudflare API endpoint, and Cloudflare then bills the buyer and pays the publisher monthly. Publishers can track estimated earnings and usage by buyer and domain in the Cloudflare dashboard, with planned future enhancements such as report context like keywords or product type.
Europe Data Residency for Atlas Embedding and Reranking
MongoDB announced the public preview of data residency in Europe and the United States for the Embedding and Reranking API Service (ERAS) on Atlas. This introduces a Geography, a named group of cloud regions, as a processing boundary for inference, distinct from the region-level cluster provisioning. Requests to a Designated Service are processed and logged only within the chosen Geography, with a 10% price premium for scoped processing. The Europe Geography initially covers EEA regions, and the US Geography covers US regions, with plans to expand. This approach supports data sovereignty goals for European organizations while maintaining a global endpoint for workloads without residency requirements.
From manual to autonomous: how AI agents are transforming electric grid operations
Electric utilities are facing rising demand, retiring generation, extreme weather, aging infrastructure, and fragmented operational data that manual processes cannot manage at scale. AI agents are presented as a human-centered alternative that synthesizes heterogeneous data, learns from outcomes, and progresses from human-approved recommendations to exception-based control and eventually autonomous operations within defined parameters. Hawaiian Electric used a Retrieval Augmented Generation proof-of-concept with Databricks AI Search, Unity Catalog, and Lakeflow Declarative Pipelines to query regulatory documents and provide page-specific citations. The system reduced response times from five minutes to five seconds and was implemented in two weeks, while the article describes broader potential for predictive maintenance, outage response, load forecasting, and customer service.
Building real-time product search on Databricks
Real-time product search for an online marketplace must retrieve, filter, rank, and respond in milliseconds while balancing latency, relevance, revenue, and click-through rate. The Databricks architecture divides the pipeline into ingestion, retrieval, and refinement, using Lakeflow pipelines and AI Functions for source processing, AI Search for embeddings and hybrid retrieval, and Lakebase for sub-10ms operational signals. It recommends validating p95/p99 latency, throughput, retrieval quality, and engagement metrics, with MLflow supporting systematic evaluation and caching helping meet strict latency requirements. As an example, FOX Sports used AI Search for thousands of QPS and achieved a 2x improvement in query success rate, combining streaming ingestion, two-phase retrieval, model serving, caching, and trending searches.
Building AI Search on Heroku
The reference architecture addresses a common RAG failure: vector search can return semantically similar documentation that does not answer the user's question. Its query pipeline retrieves the top 20 chunks with cosine similarity in Heroku Postgres and pgvector, then sends those candidates to Cohere Rerank 3.5, a cross-encoder reranker, before Claude 3.5 Sonnet generates an answer. The indexing pipeline crawls URLs, filters likely boilerplate, splits documents into 1,000-character chunks with 200-character overlap at natural boundaries, embeds them with Cohere Embed Multilingual, and batch-inserts them. Server-Sent Events stream embedding, search, reranking, sources, and generated text to avoid a blank screen during the multi-step request. The reranking stage can be made optional when lower latency matters more than precision.
Related topics
All topics →Weekly digest
The best engineering reads, once a week.