---
title: "Synthetic Data"
description: "4 posts about Synthetic Data, summarised, each linking to the original."
---

# Synthetic Data
> 4 posts about Synthetic Data, summarised, each linking to the original.

## Articles

### [Scaling Indic Pretraining: 2.8 Trillion Tokens and an Open Data Generation Pipeline](https://yomu.fyi/post/scaling-indic-pretraining-2-8-trillion-tokens-and-an-open-data-generat.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Will Jennings, Kiran Praveen, Sriharsha Niverty, Raviraj Joshi, Markus Kliegl, Yev Meyer, Utkarsh Vaidya, Niket Agarwal
- Published: Oct 16, 2026
- Type: Announcement

A joint open-source release from NVIDIA and IIT Madras’s AI4Bharat introduces Nemotron-pretraining-Indic-v1, a corpus of approximately 2.8 trillion tokens across all 22 Indian languages listed in India’s Constitution. The corpus includes native-script data for all languages and romanized text for twelve, spanning 34 language-script configurations, built from 15 curated English source subsets covering mathematics, code, and question-answer content. A GPU-accelerated pipeline using NeMo Data Designer and Gemma-4-31B-it translates documents, assesses quality with a Multidimensional Quality Metrics rubric, repairs low-scoring outputs, and converts a subset to romanized text. For ten additional languages, translation uses Indic Translate from Bodhan AI with adapted document-windowing for longer texts. Every record retains a source identifier for provenance, and the full generation recipe including prompts and quality checks is released openly so developers can adapt it to their own source material.


### [How Flo Health built safer, more useful open models with Databricks AI Runtime](https://yomu.fyi/post/how-flo-health-built-safer-more-useful-open-models-with-databricks-ai.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Vladislav Nedosekin, András Meczner, Julia Powell, Tejas Sundaresan, Michael Shtelma
- Published: Oct 12, 2026
- Type: Problem & solution

Flo Health, a women’s health app with 81 million monthly active users, aimed to improve generative AI responses for medical safety and usefulness while operating at scale. Relying on general-purpose models created cost, dependency, and domain-expertise limitations, so Flo built a post-training pipeline with Databricks AI Runtime. The team defined 26 AI judges across three categories: 12 medical-safety judges acting as hard constraints, 6 micro judges for nuanced behaviors, and 8 usefulness criteria. Using these judges as a reward signal, they generated over 15,000 synthetic questions from privacy-first principles and applied reinforcement fine-tuning with supervised fine-tuning (SFT) and direct preference optimization (DPO). After fine-tuning on 32 NVIDIA H100 GPUs using PyTorch and Fully Sharded Data Parallel (FSDP), medical safety scores rose from 34% to 61%, usefulness from 42% to 76%, and the combined metric improved nearly threefold to 47%.


### [AutoSynthData: Generating Training Data for Enterprise Agents](https://yomu.fyi/post/autosynthdata-generating-training-data-for-enterprise-agents.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Esakkivel Esakkiraja, Shruthan Radhakrishna, Denis Akhiyarov, Sagar Davasam
- Published: Oct 2, 2026
- Type: Problem & solution

AutoSynthData, built by ServiceNow CoreAI, generates training data for enterprise agents by targeting capability gaps from evaluation, using a stronger teacher such as Qwen3.8-27B to demonstrate successes. It uses a target model’s failures and a stronger teacher’s successes to decide what to learn next. The system produces tasks composing a system specification, user prompt, and verifier, ensuring feasibility, realism, and difficulty against the target environment. After sample-level checks and batch-level review, the dataset grows via target and multiply phases, generating 2,000 samples in 18 hours for the Hybrid domain. In EnterpriseOps Gym, synthetic SFT on Gemma-4-26B-A4B-it improved mean Pass@1 by 7.2 points (35% relative) in Hybrid and from 18.77% to 27.18% in ITSM, showing the approach works across domains.


### [Limits of Confidence in Diffusion](https://yomu.fyi/post/limits-of-confidence-in-diffusion.md)
- Company: [Apple](https://yomu.fyi/company/apple.md)
- Published: Oct 2, 2026
- Type: Explainer

The post analyzes limitations of confidence-based ranking in discrete diffusion, where sequences are generated by writing multiple token positions per step from per-position distributions. It shows that a step matches the training distribution only when the written positions are conditionally independent given fixed tokens, and no product of per-position distributions can match a dependent group. Additionally, per-position distributions do not determine group dependence: two joint distributions can have identical per-position marginals but differ in value combinations. On the synthetic ScanAndAdd task, every group of two or more undetermined positions written by a confidence ranking is dependent, and the generated distribution is 29 times the sampling-noise floor in total variation, with per-sample metrics at 1.0. The post also includes updates on two related readings: on-policy distillation for reasoning models and self-supervised learning with Transformers.