# Synthetic Data Generation for Financial AI Research with NVIDIA NeMo

[NVIDIA Developer Blog](https://yomu.fyi/company/nvidia-developer-blog) · Elizabeth Goodman · Jul 9, 2026

**Type:** Problem & solution

## Summary

Real-world financial natural language processing datasets often overrepresent common events like earnings while underrepresenting rarer occurrences such as credit-rating changes. To address this imbalance, an iterative generation pipeline produced over 500,000 unique headlines across 13 categories using NVIDIA NeMo Data Designer, NeMo Curator, and Nemotron 3 Nano. Instead of relying on a single large batch that yielded 65 percent near-duplicates, the workflow executes iterative cycles of category-weighted generation, rule-based filtering, global semantic deduplication, and dynamic category reweighting. Farthest-from-centroid few-shot example selection guided subsequent rounds toward novel semantic spaces across 82 total iterations. Executed over approximately six days on an eight-way NVIDIA B200 node, the resulting corpus enabled fine-tuning compact student language models to achieve 95 percent of teacher model performance.

## Context

Fine-tuning language models for financial natural language processing is constrained by limited, imbalanced real-world data that overrepresents earnings and stock price movements while underrepresenting rarer events such as credit-rating changes, product approvals, and labor issues.

## Approach / What changed

An iterative synthetic data pipeline orchestrates Nemotron 3 Nano via vLLM and NeMo Data Designer across 12 topics plus an Other category. Each cycle generates batches, applies rule-based text filters, runs global semantic deduplication against the accumulated corpus via NeMo Curator, selects farthest-from-centroid few-shot examples, and dynamically adjusts category weights.

## Takeaways

- A single-pass naive batch of 50,000 headlines resulted in 65% near-duplicates, whereas an iterative generation loop sustained yield and reached 502,536 unique headlines over 82 iterations.
- Global semantic deduplication against the full accumulated corpus, rather than local batch-level deduplication, is required to prevent cross-batch collisions as synthetic datasets scale.
- Selecting farthest-from-centroid few-shot examples with cross-iteration similarity cutoffs actively steers model generation into novel semantic regions across rounds.

**Tags:** [Data Pipelines](https://yomu.fyi/topic/data-pipelines), [LLMs](https://yomu.fyi/topic/llm), [Machine Learning](https://yomu.fyi/topic/machine-learning)

[Read original post](https://developer.nvidia.com/blog/synthetic-data-generation-for-financial-ai-research-with-nvidia-nemo)