---
title: "Shaping Product Understanding with Contrastive Reinforcement Learning"
description: "Etsy’s search and recommendation systems need product representations that capture distinguishing details—such as materials, craft, and visual style—beyond categories and titles, despite noisy listing data, varied inventory, and latency constraints. The approach fine-tunes the open-source Qwen3-VL-8B on query, engaged-listing, and non-engaged-listing triplets, generating candidate summaries and scoring them with a frozen two-tower dense retriever. Group Relative Policy Optimization (GRPO) uses the query–engaged versus query–non-engaged similarity margin as a joint reward, encouraging grounded summaries that separate alternatives rather than generic keyword stuffing. Human review found specific, accurate details beyond titles, while adding the summaries to Etsy’s Semantic Relevance model improved macro F1 by 8.7% relative to description n-grams. Online production experiments are planned."
---

# Shaping Product Understanding with Contrastive Reinforcement Learning

[Etsy](https://yomu.fyi/company/etsy) · Pat Geitner · May 26, 2026

**Type:** Problem & solution

## Summary

Etsy’s search and recommendation systems need product representations that capture distinguishing details—such as materials, craft, and visual style—beyond categories and titles, despite noisy listing data, varied inventory, and latency constraints. The approach fine-tunes the open-source Qwen3-VL-8B on query, engaged-listing, and non-engaged-listing triplets, generating candidate summaries and scoring them with a frozen two-tower dense retriever. Group Relative Policy Optimization (GRPO) uses the query–engaged versus query–non-engaged similarity margin as a joint reward, encouraging grounded summaries that separate alternatives rather than generic keyword stuffing. Human review found specific, accurate details beyond titles, while adding the summaries to Etsy’s Semantic Relevance model improved macro F1 by 8.7% relative to description n-grams. Online production experiments are planned.

## Context

Etsy’s diverse marketplace contains product details that can determine buyer relevance but are often buried in long, inconsistent listing data. Fixed structured schemas are difficult to apply across one-of-a-kind inventory, while production systems face latency constraints and cannot always process every raw signal, including images.

## Approach / What changed

Qwen3-VL-8B generates candidate natural-language summaries for query, engaged-listing, and non-engaged-listing triplets. A frozen two-tower dense retriever scores each summary against the query, and GRPO fine-tunes the generator using the margin between engaged and non-engaged similarities as a joint contrastive reward.

## Takeaways

- The training data defines engaged listings as those clicked, purchased, or favorited for a query; non-engaged listings come from shown-but-uninteracted-with results and random samples.
- The paired margin reward is designed to reject generic keyword-stuffed summaries because a summary must improve the engaged listing’s similarity while reducing the non-engaged listing’s similarity.
- Using the generated summaries in the Semantic Relevance model improved macro F1 by 8.7% relative to using description n-grams, indicating transfer beyond the retrieval model used for optimization.

**Tags:** [LLMs](https://yomu.fyi/topic/llm), [Model Evaluation](https://yomu.fyi/topic/model-evaluation), [Reinforcement Learning](https://yomu.fyi/topic/reinforcement-learning), [Search](https://yomu.fyi/topic/search)

- Source: [Etsy](https://www.etsy.com/codeascraft/shaping-product-understanding-with-contrastive-reinforcement-learning)
- Source URL: https://www.etsy.com/codeascraft/shaping-product-understanding-with-contrastive-reinforcement-learning
- Ingested by Yomu: 2026-09-09T20:03:03.113Z

[Read original post](https://www.etsy.com/codeascraft/shaping-product-understanding-with-contrastive-reinforcement-learning)