---
title: "Machine Learning in Content Moderation at Etsy"
description: "Etsy describes a supervised-learning system for detecting potentially violating listings at marketplace scale, alongside community reporting and other automated controls. Training data combines logged and annotated positives, hard negatives, and randomly sampled easy negatives, with multimodal textual and imagery signals split by time for progressive evaluation. The architecture uses BERT-based text encoding with ALBERT for reduced compute, EfficientNet image encoding, late concatenation of modality representations, a softmax output, and focal loss for class imbalance. Etsy evaluates candidates offline and in production through API-serving, canary releases, and champion-challenger A/B tests, while using proxy recall because full manual review is infeasible. Frequent retraining and monitoring support ongoing iteration; the methodology has identified and removed more than 100,000 violations, alongside other tools and services."
---

# Machine Learning in Content Moderation at Etsy

[Etsy](https://yomu.fyi/company/etsy) · David Azcona · Aug 6, 2024

**Type:** Explainer

## Summary

Etsy describes a supervised-learning system for detecting potentially violating listings at marketplace scale, alongside community reporting and other automated controls. Training data combines logged and annotated positives, hard negatives, and randomly sampled easy negatives, with multimodal textual and imagery signals split by time for progressive evaluation. The architecture uses BERT-based text encoding with ALBERT for reduced compute, EfficientNet image encoding, late concatenation of modality representations, a softmax output, and focal loss for class imbalance. Etsy evaluates candidates offline and in production through API-serving, canary releases, and champion-challenger A/B tests, while using proxy recall because full manual review is infeasible. Frequent retraining and monitoring support ongoing iteration; the methodology has identified and removed more than 100,000 violations, alongside other tools and services.

## Context

Etsy aims to keep its marketplace safe and trusted while enforcing policies and removing potentially violating or infringing items at scale. The evaluation goal is to minimize the impact on well-intentioned sellers while detecting offending listings.

## Approach / What changed

The system builds time-split multimodal datasets from annotated positives, hard negatives, and randomly sampled easy negatives. It combines BERT-based text encoding, ALBERT, EfficientNet image encoding, late multimodal concatenation, softmax classification, and focal loss, then evaluates and promotes models through offline testing, canary releases, champion-challenger A/B tests, monitoring, and retraining.

## Takeaways

- Hard negatives are annotated non-violating listings that resemble violations, while easy negatives are randomly sampled items used to improve generalizability.
- The model uses separate text and image encoders, constrains each modality's representation before late concatenation, and applies focal loss to address class imbalance.
- Because manually reviewing millions of new listings per day is infeasible, recall is estimated with an annotated-data proxy and production performance is assessed through canary releases and A/B tests.

**Tags:** [Machine Learning](https://yomu.fyi/topic/machine-learning), [Model Evaluation](https://yomu.fyi/topic/model-evaluation), [Multimodal](https://yomu.fyi/topic/multimodal)

- Source: [Etsy](https://www.etsy.com/codeascraft/machine-learning-in-content-moderation-at-etsy)
- Source URL: https://www.etsy.com/codeascraft/machine-learning-in-content-moderation-at-etsy
- Ingested by Yomu: 2026-09-09T16:03:29.772Z

[Read original post](https://www.etsy.com/codeascraft/machine-learning-in-content-moderation-at-etsy)