Build a Domain-Specific Embedding Model in Under a Day

Summary

The tutorial presents a pipeline for adapting a general-purpose embedding model when it misses fine-grained distinctions in domain documents and labeled query–document data is unavailable. It generates synthetic question–answer pairs with NeMo Data Designer, filters them for quality, mines hard negatives with a 95% positive-score margin ceiling, and unrolls multi-hop queries for contrastive training. The recipe fine-tunes Llama-Nemotron-Embed-1B-v2, evaluates base and adapted models with BEIR metrics, and exports the result to ONNX or TensorRT. On the reported datasets, it improved Recall@10 and NDCG@10 by more than 10%; Atlassian increased Recall@60 from 0.751 to 0.951 on JIRA data. The complete workflow can run in roughly 2–3 hours for about 500 documents, using a single 80GB Ampere-or-newer GPU.

Takeaways

1. 1

   Hard-negative mining embeds queries and passages, masks labeled positives, excludes candidates above 95% of the minimum positive score, and selects the top five surviving non-positive passages by default.

2. 2

   Synthetic questions support configurable complexity levels from 2–5 and hop counts from 1–3; quality scoring covers relevance, accuracy, context support, and clarity before pairs enter training.

3. 3

   NVIDIA NIM exposes the exported model through an OpenAI-compatible /v1/embeddings endpoint, while a BEIR-based verification step checks for accuracy changes after ONNX or TensorRT conversion.