---
title: "Search"
description: "185 posts about Search, summarised, each linking to the original."
---

# Search
> 185 posts about Search, summarised, each linking to the original.

## Articles

### [Clustering billions of products for agentic commerce with Catalog API](https://yomu.fyi/post/clustering-billions-of-products-for-agentic-commerce-with-catalog-api.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Jun 17, 2026
- Type: Explainer

Shopify’s Catalog must reconcile billions of listings whose merchants use different product and variant structures, so AI shopping agents can identify the same real-world product. Product clustering groups related listings under a Universal Product Identifier, with precision prioritized over recall because wrong products can lead to incorrect purchases while missed variants remain findable. The intra-store system starts with rules-based singleton detection, then uses ANN and sparse average linkage for pre-chunking, and applies a two-stage LLM process that proposes clusters and critiques them using the core value proposition framework. Dynamic per-chunk strict JSON schemas enforce complete product coverage, support arbitrary chunking, and improved recall by 8% on the Toloka evaluation dataset in one schema change. The resulting pipeline is presented as a scalable intra-store foundation, while cross-store clustering into global UPIs is identified as the next step.


### [Agentic Resource Discovery: Let agents search](https://yomu.fyi/post/agentic-resource-discovery-let-agents-search.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: ben burtenshaw, shaun smith
- Published: Jun 17, 2026
- Type: Explainer

Agentic Resource Discovery (ARD) is a draft, open specification for discovering agent capabilities at runtime, addressing a model in which users and developers must pre-install, integrate, and maintain each MCP tool, skill, or A2A agent. It defines a static ai-catalog.json manifest and a dynamic POST /search registry API, using indexed signals such as publisher identity, representative queries, compliance attestations, and tags to move capability selection outside the LLM. Hugging Face's Discover Tool adapts Hub semantic search into ARD catalog entries for running Spaces, exposing skills, MCP servers, or raw Space metadata according to the requested media type and wrapping agents.md files into installable SKILL.md artifacts. The tool is available through the hf CLI, REST, and MCP, while planned work includes federation modes and Hub support for profile-level manifests.


### [Announcing Lakebase Search: agent-native retrieval built into Lakebase Postgres](https://yomu.fyi/post/announcing-lakebase-search-agent-native-retrieval-built-into-lakebase.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Pranav Aurora, Zhou Sun, Jinjing Zhou
- Published: Jun 16, 2026
- Type: Announcement

Lakebase Search is a beta offering on AWS and Azure that adds hybrid vector and full-text retrieval to Lakebase Postgres. It uses the lakebase\_vector and lakebase\_text extensions to keep retrieval, memory, operational data, and hybrid search in one backend. lakebase\_vector retains pgvector types and operators, applies RaBitQ clustering and compression for 32x smaller indexes, and targets more than 1B vectors, while lakebase\_text replaces GIN with object-storage-optimized BM25 ranking. A tiered cache keeps hot data on NVMe and places colder data in object storage; the source reports lower memory needs, faster index builds, and cold-cache startup than standard pgvector HNSW in its LAION-100M benchmark. The extensions also combine vector similarity and keyword relevance with reciprocal rank fusion in a single SQL query, enabling joins and tenant filtering alongside transactional workflows.


### [Accelerate search queries with full-text search indexes on Databricks](https://yomu.fyi/post/accelerate-search-queries-with-full-text-search-indexes-on-databricks.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Yu Xu, Yingyi Bu, Ivan Vezilić
- Published: Jun 16, 2026
- Type: Announcement

Databricks introduces full-text search indexes in Beta on Databricks Runtime 18.2 to accelerate substring and keyword queries on large open-format tables without changing their layouts. The indexes tokenize text columns into a compact lookup structure mapping tokens to matching rows; at query time, the engine uses it to identify candidate files and skip most of the table. They are maintained asynchronously, require no query hints, preserve complete results when stale by scanning indexed and non-indexed data as needed, and support Unity Catalog managed Delta and Iceberg tables on serverless and classic compute. A Trust and Safety team reported a substring search running more than 100x faster on a petabyte-scale table, while Liquid clustering remains complementary because it optimizes column-value filters rather than text within fields.


### [What a difference 10 years of open source makes](https://yomu.fyi/post/what-a-difference-10-years-of-open-source-makes.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Published: Jun 16, 2026
- Type: Explainer

ClickHouse marks ten years since becoming open source under the Apache 2.0 license, crediting more than 2,600 contributors and a decade of monthly releases for its growth into one of the world’s most widely deployed analytical databases. Recent engineering work spans joins, updates and deletes, JSON, full-text search, vector search, query acceleration, integrations, and developer tooling. Join-focused changes made the TPC-H SF100 workload 26× faster between versions 22.4 and 26.4, while statistics-based reordering reduced one six-table query from 3,903.7 seconds to 2.7 seconds. The project also publishes open benchmarks, including ClickBench, JSONBench, CostBench, and PostgresBench, with scripts, configurations, data, and results intended for independent validation. Its next phase continues to emphasize community contributions and broader ways to use ClickHouse.


### [What is enterprise intelligence?](https://yomu.fyi/post/what-is-enterprise-intelligence.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Databricks Staff
- Published: Jun 12, 2026
- Type: Explainer

Enterprise intelligence (EI) is presented as an organization-wide capability combining business intelligence, knowledge management, enterprise search and AI to turn structured and unstructured data into decisions and actions. Unlike traditional BI, which centers on dashboards, reports and structured data, EI connects these capabilities through a shared architecture and governed business context. The described stack includes a lakehouse-based data foundation, batch and streaming pipelines, governance, semantics, analytics, search, machine learning, generative AI and a decision layer. Shared definitions such as “active customer” and “monthly revenue” are intended to keep dashboards, queries and AI agents aligned, while maintained context addresses knowledge that becomes stale as the business changes. The result described is a common trusted source from which people, applications and agents can produce insights and initiate actions.


### [How Dropbox uses MCP and Dash to close the design-to-code security gap](https://yomu.fyi/post/how-dropbox-uses-mcp-and-dash-to-close-the-design-to-code-security-gap.md)
- Company: [Dropbox](https://yomu.fyi/company/dropbox.md)
- Author: Ilya Yakovlev,Andrew Cheung,Binoy Dash,Simran Jumani,Dmitriy Meyerzon,Mark Breitenbach,Ishan Mishra
- Published: Jun 12, 2026

Dropbox developed a system using the Model Context Protocol (MCP) and Dash's semantic search to bridge the gap between security threat models and code implementation. By retrieving original security documents during pull requests, an LLM agent automatically evaluates whether the proposed code adheres to previously agreed-upon security requirements. This approach surfaces design regressions and missing controls that traditional static analysis tools miss.


### [Production-Ready Agents Need A Production-Ready Data Platform](https://yomu.fyi/post/production-ready-agents-need-a-production-ready-data-platform.md)
- Company: [MongoDB](https://yomu.fyi/company/mongodb.md)
- Author: Pablo Stern
- Published: Jun 11, 2026
- Type: Explainer

AI development teams face constant shifts in model providers and agent frameworks, demanding data platforms that provide scalable, real-time context management. Agentic workloads require blending unstructured enterprise data, short-term session state, and persistent long-term memory. MongoDB addresses these requirements through its native JSON document model and integrated retrieval capabilities, combining full-text search, vector search, and hybrid search directly over operational data. Customer implementations such as DevRev, ElevenLabs, and Adobe rely on Atlas to achieve sub-100 millisecond hybrid retrieval and handle billions of requests. Additionally, MongoDB is collaborating with LangChain and ecosystem partners to establish open reference architectures and shared interfaces for portable agent memory across frameworks.


### [Datadog and ClickHouse partner to bring full-fidelity data to modern observability](https://yomu.fyi/post/datadog-and-clickhouse-partner-to-bring-full-fidelity-data-to-modern-o.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Published: Jun 10, 2026
- Type: Announcement

ClickHouse and Datadog announce a partnership aimed at reducing the tradeoff between retaining complete telemetry and keeping it accessible through familiar observability workflows. New Preview capabilities let organizations route logs directly to ClickHouse through Datadog Observability Pipelines and search those logs from the Datadog Log Explorer. Pipelines can parse, enrich, filter, transform, and redact data before storage, while ClickHouse Cloud’s separation of storage and compute supports long-term retention, independently scaled query resources, and workload isolation. Federated log search queries data directly in ClickHouse, avoiding duplicate copies or re-ingestion while allowing high-volume logs to remain searchable and historical investigations to use full-fidelity telemetry. The partnership combines Datadog’s collection and investigation experience with ClickHouse’s storage and analytics foundation; the native integration and federated search are available in Preview.


### [What is AI Search?](https://yomu.fyi/post/what-is-ai-search.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Databricks Staff
- Published: Jun 4, 2026
- Type: Explainer

AI search interprets natural-language questions using semantic understanding and large language models, then returns synthesized answers grounded in cited sources. Unlike traditional search, which matches query words to indexed pages, it tracks context and intent for conversational follow-ups. Its pipeline includes query understanding, embeddings, vector search, retrieval, LLM synthesis, and citations; retrieval-augmented generation (RAG) connects generated responses to source material. The article distinguishes consumer tools that search the open web from enterprise systems that retrieve private, governed company data while respecting access permissions. It concludes that AI search can improve direct information access but should not be treated as definitive, and that sensitive business use requires purpose-built platforms with governance, security, and retrieval quality.


### [3x Faster Search: Parallel Test-Time Scaling with Instructed-Retriever-1](https://yomu.fyi/post/3x-faster-search-parallel-test-time-scaling-with-instructed-retriever.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: The Databricks AI Research Team
- Published: Jun 4, 2026
- Type: Announcement

Instructed-Retriever-1 is introduced as the retrieval-specialized model behind an update to Agent Bricks Knowledge Assistant, targeting search quality and latency. It parallelizes query and filter generation for broader candidate recall, then applies multi-pivot groupwise reranking to improve precision, using one model for both retrieval stages. The model was trained on synthetic enterprise-style environments and evaluated on realistic Knowledge Assistant workloads, where it scored 81.0 nDCG@10 for reranking and matched Claude Sonnet 4.5 retrieval quality on KARLBench. Serving uses a Mixture-of-Experts architecture, FP8 quantization, speculative decoding, and infrastructure tuning; the update reports more than 3x faster search, 2x faster answer generation, and TTFT around two seconds. End-to-end latency was consistently below 10 seconds in the offline evaluation setup.


### [Agentic Supplier Management with MongoDB Atlas, Voyage AI, and Multi-Modal Search](https://yomu.fyi/post/agentic-supplier-management-with-mongodb-atlas-voyage-ai-and-multi-mod.md)
- Company: [MongoDB](https://yomu.fyi/company/mongodb.md)
- Author: Ronan Conlon
- Published: Jun 3, 2026
- Type: Problem & solution

Retailers face major operational bottlenecks when managing supply chain disruptions through legacy enterprise resource planning systems and siloed documents. Traditional tabular structures struggle to index multi-modal data like images and PDF contracts, delaying critical supplier discovery during regional crises or tariff changes. To overcome these constraints, organizations decouple supplier management into a dedicated operational data layer using MongoDB Atlas and Voyage AI. This architecture stores polymorphic supplier profiles and high-dimensional vector embeddings within a unified collection, enabling natural language semantic search for alternate partners. Real-time updates propagate across systems using MongoDB Change Streams, allowing businesses to rapidly assess supply chain impacts and identify alternative vendors in minutes.


### [From decentralized Docs-as-Code to a centralized repository: Evolving Grab's documentation strategy](https://yomu.fyi/post/from-decentralized-docs-as-code-to-a-centralized-repository-evolving-g.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Karen Kue
- Published: May 29, 2026
- Type: Explainer

Grab adopted Docs-as-Code in early 2021 to bring technical documentation into engineers’ code workflows, addressing scattered sources, version confusion, discoverability, and quality gaps. Documentation was written in Markdown, stored in version-controlled repositories, reviewed through merge requests, checked in CI/CD, and surfaced through an internal developer portal. As engineering scaled, decentralized repositories produced inconsistent structures and terminology, unreliable organization-wide quality automation, aging content, and tracking problems when source locations changed. Grab therefore moved to a centralized repository while retaining Docs-as-Code, adding shared templates and linters, unified search through fuzzy title lookup and Glean, automated synchronization during migration, and training and feedback programs. The post concludes that decentralized, centralized, or hybrid models fit different contexts, with centralization emphasizing discoverability, governance, consistency, ownership metadata, and analytics.


### [Beyond the Menu Tree: How Yelp Built a Smarter Customer Success Chatbot with AI](https://yomu.fyi/post/beyond-the-menu-tree-how-yelp-built-a-smarter-customer-success-chatbot.md)
- Company: [Yelp](https://yomu.fyi/company/yelp.md)
- Author: Lina Lee, Machine Learning Engineer; Nelson Lee, Engineering Manager
- Published: May 27, 2026
- Type: Problem & solution

Yelp transitioned from a legacy customer support chatbot that relied on rigid phrase matching and static menus to a retrieval-augmented generation system. The updated architecture classifies incoming queries using a large language model to route users into one of five specialized workflows, including QA, Billing, Refund, Cancel, and Review. For general QA, the system uses FAISS to search an in-memory vectorstore built exclusively from article metadata such as titles, summaries, and headers rather than raw article text. Retrieved articles are supplied to the language model as prompt context, followed by automated validations for safety, character limits, and hallucinated hyperlinks. In production A/B testing, this metadata-driven design doubled the chatbot resolution rate while achieving approximately 94% recall@5.


### [Shaping Product Understanding with Contrastive Reinforcement Learning](https://yomu.fyi/post/shaping-product-understanding-with-contrastive-reinforcement-learning.md)
- Company: [Etsy](https://yomu.fyi/company/etsy.md)
- Author: Pat Geitner
- Published: May 26, 2026
- Type: Problem & solution

Etsy’s search and recommendation systems need product representations that capture distinguishing details—such as materials, craft, and visual style—beyond categories and titles, despite noisy listing data, varied inventory, and latency constraints. The approach fine-tunes the open-source Qwen3-VL-8B on query, engaged-listing, and non-engaged-listing triplets, generating candidate summaries and scoring them with a frozen two-tower dense retriever. Group Relative Policy Optimization (GRPO) uses the query–engaged versus query–non-engaged similarity margin as a joint reward, encouraging grounded summaries that separate alternatives rather than generic keyword stuffing. Human review found specific, accurate details beyond titles, while adding the summaries to Etsy’s Semantic Relevance model improved macro F1 by 8.7% relative to description n-grams. Online production experiments are planned.


### [Introducing the Ettin Reranker Family](https://yomu.fyi/post/introducing-the-ettin-reranker-family.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Tom Aarsen
- Published: May 19, 2026
- Type: Announcement

Tom Aarsen introduces six Sentence Transformers CrossEncoder rerankers, ranging from 17M to 1B parameters, alongside their training data and full recipe. Each model uses an Ettin ModernBERT encoder with unpadded attention, RoPE, GeGLU, up to 8,192 tokens of context, and a modular classification head whose ablations favored CLS pooling over mean pooling. Training applies pointwise MSE distillation to scores from mixedbread-ai/mxbai-rerank-large-v2 using a broad and retrieval-focused dataset mixture. The conclusion reports state-of-the-art results at every released size, with the 400M model within 0.0024 and the 1B model within 0.0001 of the 1.54B teacher on MTEB, while all models outperform the ms-marco-MiniLM-L\*-v2 family on MTEB and NanoBEIR.


### [Operator: A look under the hood](https://yomu.fyi/post/operator-a-look-under-the-hood.md)
- Company: [Intercom](https://yomu.fyi/company/intercom.md)
- Author: Jack Ryan
- Published: May 15, 2026
- Type: Explainer

Operator is a customer-operations Agent built to help teams understand, manage, and improve customer experience across workspaces. The post distinguishes a production Agent from a prototype that simply gives an LLM API access and a system prompt, describing more than 50 purpose-built tools and 10 skills for data retrieval, reporting, investigation, and operational tasks. Its intelligence layer combines semantic search tuned against millions of support conversations, workspace-specific attribute awareness, and encoded reasoning, while the action layer supports configuration and content changes through reviewable diffs requiring explicit approval. A hybrid conversational and graphical interface presents charts, dashboards, and structured proposals within the existing platform. The post argues that read-only custom tools can be built in weeks, but production reliability, safe actions, multi-tenant isolation, graceful degradation, and ongoing maintenance require substantially greater investment.


### [Granite Embedding Multilingual R2: Open Apache 2.0 Multilingual Embeddings with 32K Context — Best Sub-100M Retrieval Quality](https://yomu.fyi/post/granite-embedding-multilingual-r2-open-apache-2-0-multilingual-embeddi.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Radu Florian, Parul Awasthy, Aashka Trivedi, Madison Lee
- Published: May 14, 2026
- Type: Announcement

Granite Embedding Multilingual R2 introduces two Apache 2.0 multilingual embedding models built on ModernBERT: a 97M-parameter compact model and a 311M full-size model. Both cover more than 200 languages, receive enhanced retrieval training for 52 languages and nine programming languages, and support context lengths up to 32,768 tokens, compared with 512 for the R1 models. The compact model scores 60.3 on MTEB Multilingual Retrieval, while the full-size model scores 65.2; the reported results place them among the strongest open models in their parameter ranges. R2 replaces the XLM-RoBERTa architecture with alternating attention lengths, rotary position embeddings, newer multilingual and code tokenizers, and Flash Attention 2.0 support, while the 311M model adds Matryoshka embeddings and both models provide ONNX and OpenVINO weights.


### [Are you winning in AI search? Do this self-assessment.](https://yomu.fyi/post/are-you-winning-in-ai-search-do-this-self-assessment.md)
- Company: [Brandwatch](https://yomu.fyi/company/brandwatch.md)
- Author: Ksenia Newton
- Published: May 13, 2026
- Type: Tutorial

AI search is reshaping consumer discovery, but the post says most brands do not know what AI platforms say about them. It presents a four-step assessment: choose one country, define a focused category, identify roughly 100 consumer questions, and test whether the brand appears in answers from ChatGPT, Gemini, Perplexity, and Google AI Overviews. Search Intelligence platforms track queries, mentions, recommendations, citations, and content gaps across search and AI tools. The post recommends answering real queries with dedicated content, earning coverage from accessible independent sources, participating helpfully in online conversations, and monitoring AI answers monthly because they change. Its examples report a 25% reduction in Orange call-center contacts, 50% lower Campari costs, and Aromazon reaching €40 million in annual revenue.


### [Full-screen Tabs and Compact Chats](https://yomu.fyi/post/full-screen-tabs-and-compact-chats.md)
- Company: [Cursor](https://yomu.fyi/company/cursor.md)
- Published: May 13, 2026
- Type: Announcement

This release introduces quality-of-life improvements to the Agents Window, centered on full-screen work areas and more compact agent conversations. Full screen expands files, changes, canvases, PRs, browsers, or terminals across the right panel, replaces chat with a floating prompt bar, and is available through the panel control, command palette, or Cmd/Ctrl+Shift+M. Compact chats offer Compact, Balanced, and Detailed tool-call density settings, ranging from concise results with minimal traces to near-complete step-by-step context. The release also improves PR review visibility, scrolling, streaming, context persistence, MCP authentication, model selection, search, and task status, while fixing reliability issues across PRs, background and cloud agents, MCP/OAuth, shortcuts, hooks, model routing, webhooks, and mirrored repositories.


[Newer posts](https://yomu.fyi/topic/search/page/3.md) · [Older posts](https://yomu.fyi/topic/search/page/5.md)