Clustering billions of products for agentic commerce with Catalog API

Summary

Shopify’s Catalog must reconcile billions of listings whose merchants use different product and variant structures, so AI shopping agents can identify the same real-world product. Product clustering groups related listings under a Universal Product Identifier, with precision prioritized over recall because wrong products can lead to incorrect purchases while missed variants remain findable. The intra-store system starts with rules-based singleton detection, then uses ANN and sparse average linkage for pre-chunking, and applies a two-stage LLM process that proposes clusters and critiques them using the core value proposition framework. Dynamic per-chunk strict JSON schemas enforce complete product coverage, support arbitrary chunking, and improved recall by 8% on the Toloka evaluation dataset in one schema change. The resulting pipeline is presented as a scalable intra-store foundation, while cross-store clustering into global UPIs is identified as the next step.

Context

Shopify hosts billions of product listings across millions of stores, but merchants structure products and variants differently. AI shopping agents therefore need to recognize when differently modeled listings represent the same real-world product. Clustering errors create precision failures, where distinct products are grouped together, or recall failures, where valid variants are missed.

Approach / What changed

The system prioritizes precision by setting a hard precision threshold and maximizing recall within it. For intra-store clustering, it uses singleton detection, ANN and sparse average-linkage pre-chunking, and a two-stage LLM flow guided by the core value proposition framework. Dynamic strict JSON schemas generated per chunk enforce complete product classification, while normalized product IDs improve cache reuse and reduce costs.

Takeaways

• The core value proposition test treats an attribute as a variant when it does not change what the buyer is primarily purchasing; otherwise, that attribute contributes to product identity and can split products into separate UPIs.
• Dynamic structured-output schemas require every input product ID to appear in the response, making arbitrary chunking viable without complex reconciliation for omitted products.
• Schema design affects clustering quality: removing non-ASCII tokens from the output structure improved recall by 8% on the Toloka evaluation dataset, while misaligned title formatting reduced recall on the GTX dataset.

Related reading