# Using LLMs to amplify human labeling and improve Dash search relevance

[Dropbox](https://yomu.fyi/company/dropbox) · Ilya Yakovlev,Andrew Cheung,Binoy Dash,Simran Jumani,Dmitriy Meyerzon,Mark Breitenbach,Ishan Mishra,Kazuaki Okumura,Mike White,Kevin Altschuler,Facundo Agriel,Ishan Mishra,Eric Wang,Dmitriy Meyerzon,Dmitriy Meyerzon · Feb 26, 2026

## Summary

Dropbox Dash uses large language models (LLMs) to amplify human labeling efforts for training its search relevance and ranking models. By validating and optimizing LLM evaluators against a small set of human-labeled internal data, Dropbox creates massive, high-quality training datasets offline for production rankers like XGBoost without incurring high latency or latency costs at query time.

## Takeaways

- Offline LLM evaluators act as teacher models that generate millions of relevance labels to train fast, lightweight ranking models like XGBoost.
- Providing LLMs with tool access helps resolve company-specific acronyms and context that standard query-document pairs lack.
- Prompt optimization frameworks like DSPy and discrepancy sampling (comparing user clicks/skips to LLM scores) efficiently drive down evaluation error against human baselines.

**Tags:** [LLMs](https://yomu.fyi/topic/llm), [Machine Learning](https://yomu.fyi/topic/machine-learning), [Search](https://yomu.fyi/topic/search)

[Read original post](https://dropbox.tech/machine-learning/llm-human-labeling-improving-search-relevance-dropbox-dash)