---
title: "Leveraging multimodal LLMs for Shopify’s global catalogue: Recap of expo talk at ICLR 2025"
description: "Shopify’s Global Catalogue addresses fragmented, multilingual, multimodal product data that makes semantic search, faceting, matching, and AI-driven discovery difficult across millions of merchants. The system combines product data, understanding, matching, and reconciliation layers to standardize metadata, identify duplicate listings, and create canonical records with Universal Product IDs. Shopify fine-tunes vision large language models for multiple tasks per catalogue entity, while streaming infrastructure processes more than 10 million product updates daily and serves 40 million multimodal LLM-powered inferences. The resulting data supports merchant-admin suggestions, search, recommendations, embeddings, and conversational commerce, although scalability, accuracy, latency, model consolidation, graph reasoning, and continuous retraining remain active challenges."
---

# Leveraging multimodal LLMs for Shopify’s global catalogue: Recap of expo talk at ICLR 2025

[Shopify](https://yomu.fyi/company/shopify) · 2023-10-18 · Jul 16, 2025

**Type:** Explainer

## Summary

Shopify’s Global Catalogue addresses fragmented, multilingual, multimodal product data that makes semantic search, faceting, matching, and AI-driven discovery difficult across millions of merchants. The system combines product data, understanding, matching, and reconciliation layers to standardize metadata, identify duplicate listings, and create canonical records with Universal Product IDs. Shopify fine-tunes vision large language models for multiple tasks per catalogue entity, while streaming infrastructure processes more than 10 million product updates daily and serves 40 million multimodal LLM-powered inferences. The resulting data supports merchant-admin suggestions, search, recommendations, embeddings, and conversational commerce, although scalability, accuracy, latency, model consolidation, graph reasoning, and continuous retraining remain active challenges.

## Context

Shopify merchants provide product information in highly varied formats, schemas, languages, and media types. Missing, misplaced, inconsistent, or sparse attributes make products harder for machines to understand, compare, match, and surface through search, recommendations, and AI-driven shopping experiences.

## Approach / What changed

The Global Catalogue uses four integrated layers: a streaming product-data foundation, multimodal product understanding, multi-stage product matching, and reconciliation into canonical records. Vision LLMs perform multiple tasks for catalogue entities, while taxonomy governance, candidate generation, discriminator models, graph-based clustering, GPU serving, Kafka dataflows, FP8 quantization, key-value caching, and selective field prompting support large-scale processing and inference.

## Takeaways

- Multi-task, multi-entity vision LLMs perform classification, attribute extraction, image understanding, title standardization, description analysis, and review summarization; the post reports that task interdependencies improve inference quality over siloed models.
- Product matching combines locality-sensitive hashing, embeddings, probabilistic methods, deterministic signals, and discriminator-based edge pruning before connected components receive Universal Product IDs.
- Selective field prompting reduced median latency from 2 seconds to 500 milliseconds and GPU token usage by 40 percent, while FP8 quantization enabled larger batch sizes without sacrificing inference accuracy.

**Tags:** [LLMs](https://yomu.fyi/topic/llm), [Machine Learning](https://yomu.fyi/topic/machine-learning), [Recommendation Systems](https://yomu.fyi/topic/recommendation-systems), [Search](https://yomu.fyi/topic/search), [Streaming](https://yomu.fyi/topic/streaming)

- Source: [Shopify](https://shopify.engineering/leveraging-multimodal-llms)
- Source URL: https://shopify.engineering/leveraging-multimodal-llms
- Ingested by Yomu: 2026-08-30T13:19:25.407Z

[Read original post](https://shopify.engineering/leveraging-multimodal-llms)