Using Rich Image and Text Data to Categorize Products at Scale

Summary

Shopify revisited product categorization as its platform expanded to millions of merchants selling billions of products across diverse industries. The model uses product titles, descriptions, types, vendors, collections, tags, and images, converting them into embeddings with Multilingual BERT and MobileNet-V2 before hierarchical classification across the seven-level Google Product Taxonomy. Each taxonomy level is treated as a separate multi-class task, with parent predictions fed into child predictions and weighted losses combined during training; confidence thresholds filter predictions at each level. The upgraded system increased leaf precision by 8% while almost doubling coverage, although thresholds require balancing accuracy, coverage, merchant experience, and performance in sensitive categories, and future work includes addressing data imbalance and adding merchant-level features.

Takeaways

1. 1

   The categorization model uses both text and visual features rather than relying only on product text.

2. 2

   Seven taxonomy levels are modeled as separate multi-class tasks, with parent predictions feeding subsequent child-level predictions.

3. 3

   The upgraded model increased leaf precision by 8% while almost doubling coverage; thresholds were manually tuned for sensitive categories such as Religious and Ceremonial.