Shaping Product Understanding with Contrastive Reinforcement Learning

Summary

Etsy’s search and recommendation systems need product representations that capture distinguishing details—such as materials, craft, and visual style—beyond categories and titles, despite noisy listing data, varied inventory, and latency constraints. The approach fine-tunes the open-source Qwen3-VL-8B on query, engaged-listing, and non-engaged-listing triplets, generating candidate summaries and scoring them with a frozen two-tower dense retriever. Group Relative Policy Optimization (GRPO) uses the query–engaged versus query–non-engaged similarity margin as a joint reward, encouraging grounded summaries that separate alternatives rather than generic keyword stuffing. Human review found specific, accurate details beyond titles, while adding the summaries to Etsy’s Semantic Relevance model improved macro F1 by 8.7% relative to description n-grams. Online production experiments are planned.

Takeaways

1. 1

   The training data defines engaged listings as those clicked, purchased, or favorited for a query; non-engaged listings come from shown-but-uninteracted-with results and random samples.

2. 2

   The paired margin reward is designed to reject generic keyword-stuffed summaries because a summary must improve the engaged listing’s similarity while reducing the non-engaged listing’s similarity.

3. 3

   Using the generated summaries in the Semantic Relevance model improved macro F1 by 8.7% relative to using description n-grams, indicating transfer beyond the retrieval model used for optimization.