---
title: "AI4Bharat"
description: "1 posts about AI4Bharat, summarised, each linking to the original."
---

# AI4Bharat
> 1 posts about AI4Bharat, summarised, each linking to the original.

## Articles

### [Scaling Indic Pretraining: 2.8 Trillion Tokens and an Open Data Generation Pipeline](https://yomu.fyi/post/scaling-indic-pretraining-2-8-trillion-tokens-and-an-open-data-generat.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Will Jennings, Kiran Praveen, Sriharsha Niverty, Raviraj Joshi, Markus Kliegl, Yev Meyer, Utkarsh Vaidya, Niket Agarwal
- Published: Oct 16, 2026
- Type: Announcement

A joint open-source release from NVIDIA and IIT Madras’s AI4Bharat introduces Nemotron-pretraining-Indic-v1, a corpus of approximately 2.8 trillion tokens across all 22 Indian languages listed in India’s Constitution. The corpus includes native-script data for all languages and romanized text for twelve, spanning 34 language-script configurations, built from 15 curated English source subsets covering mathematics, code, and question-answer content. A GPU-accelerated pipeline using NeMo Data Designer and Gemma-4-31B-it translates documents, assesses quality with a Multidimensional Quality Metrics rubric, repairs low-scoring outputs, and converts a subset to romanized text. For ten additional languages, translation uses Indic Translate from Bodhan AI with adapted document-windowing for longer texts. Every record retains a source identifier for provenance, and the full generation recipe including prompts and quality checks is released openly so developers can adapt it to their own source material.