---
title: "Categorizing Products at Scale"
description: "Shopify describes a product-categorization system for more than one billion products across the Google Product Taxonomy, which contains over 5,000 hierarchical categories. The model uses product titles, descriptions, collections, tags, vendors, and merchant-provided types, with text preprocessing and a fixed-length term-frequency HashingTF featurizer in PySpark rather than image features or vocabulary-based methods such as Word2Vec. Kesler’s Construction transforms multiclass training into a binary dataset by embedding each candidate class into feature tokens, while logistic regression provides a single, interpretable classifier that can use taxonomy relationships and reduce computational load. The resulting baseline supports more than 20 teams and downstream experiences, while schematized Kafka events and an internal annotation platform provide human feedback; the post identifies class imbalance, multilingual text, and image features as improvement areas."
---

# Categorizing Products at Scale

[Shopify](https://yomu.fyi/company/shopify) · 2023-10-18 · Apr 30, 2020

**Type:** Explainer

## Summary

Shopify describes a product-categorization system for more than one billion products across the Google Product Taxonomy, which contains over 5,000 hierarchical categories. The model uses product titles, descriptions, collections, tags, vendors, and merchant-provided types, with text preprocessing and a fixed-length term-frequency HashingTF featurizer in PySpark rather than image features or vocabulary-based methods such as Word2Vec. Kesler’s Construction transforms multiclass training into a binary dataset by embedding each candidate class into feature tokens, while logistic regression provides a single, interpretable classifier that can use taxonomy relationships and reduce computational load. The resulting baseline supports more than 20 teams and downstream experiences, while schematized Kafka events and an internal annotation platform provide human feedback; the post identifies class imbalance, multilingual text, and image features as improvement areas.

## Context

Shopify needed to identify similar products and categorize more than one billion products despite highly varied descriptions, a taxonomy with over 5,000 hierarchical categories, and the added complexity of a tree-like category structure.

## Approach / What changed

The system uses selected text attributes, preprocessing, and PySpark’s fixed-length term-frequency HashingTF featurizer. Kesler’s Construction converts the multiclass task into binary training data by prepending candidate classes to feature tokens, followed by a single logistic regression classifier. Schematized Kafka events and an in-house annotation platform add human feedback for model updates.

## Takeaways

- HashingTF provides fixed-length numeric features that scale to any vocabulary size, though the approach may miss richer semantic representations available from methods such as Word2Vec.
- Kesler’s Construction embeds taxonomy classes into feature vectors, allowing parent-category information to influence child-category classification while using one model for thousands of classes.
- The system treats misclassification feedback as an ongoing data input through Kafka events and human annotation rather than making 100% accuracy the gold standard.

**Tags:** [Kafka](https://yomu.fyi/topic/kafka), [Machine Learning](https://yomu.fyi/topic/machine-learning), [Spark](https://yomu.fyi/topic/spark)

- Source: [Shopify](https://shopify.engineering/categorizing-products-at-scale)
- Source URL: https://shopify.engineering/categorizing-products-at-scale
- Ingested by Yomu: 2026-08-31T01:12:53.214Z

[Read original post](https://shopify.engineering/categorizing-products-at-scale)