This work introduces Nemotron-Pretraining-Indic-v1, a synthetic pretraining corpus of approximately 2.83 trillion tokens covering all 22 scheduled Indian languages in native script and twelve in romanized form and introduces the Indic-LLM-Eval-Suite, a multilingual benchmark suite for evaluating Indic language models across chat, instruction following, reasoning, knowledge, code, and translation.
Abstract
Building capable language models for India requires scalable multilingual data and robust evaluation. We introduce Nemotron-Pretraining-Indic-v1, a synthetic pretraining corpus of approximately 2.83 trillion tokens covering all 22 scheduled Indian languages in native script and twelve in romanized form. It is the largest openly released Indic pretraining corpus. Using our GPU-accelerated pipeline, we translate curated English documents, assess translation quality, and repair low-quality outputs while preserving technical content and document structure and maintaining source traceability. Through continued-pretraining experiments across twelve languages, we show that combining our synthetic corpus with natural Indic text improves knowledge and reasoning over a public-corpus baseline. Additional experiments show how data volume, script coverage, and Indic training share affect English performance. We also introduce the Indic-LLM-Eval-Suite, a multilingual benchmark suite for evaluating Indic language models across chat, instruction following, reasoning, knowledge, code, and translation. The suite combines existing Indic benchmarks with task-aware translations of English benchmarks, using automated quality assessment, targeted correction, and native-speaker review in a human-in-the-loop pipeline. We use the suite to comprehensively benchmark contemporary open-weight Indic and multilingual models across capabilities and languages, revealing substantial variation across tasks and languages. Together, these resources provide a scalable foundation for developing and evaluating more capable Indic language models.
Recent advances in large language model (LLM) pretraining highlight the role of high-quality training data in improving performance. While model-based filtering has proven effective in selecting high-quality subsets from web-scale corpora, especially for high-resource languages, low-resource languages face challenges d...
Vinko Sabolcec, Bettina Messmer, Yassine Turki et al.· 0 citations
This work combines vocabulary surgery for tokenizer adaptation without full retraining, cross-lingual transfer of quality classifiers via translation, enabling filtering without target-language annotations, and generation of instruction data through translation, task conversion, and targeted synthesis.
Yurii Paniv, Bohdan Didenko, Mykola Haltiuk et al.· 4 citations
Results suggest that Indic-specialized adaptive segmentation can reduce tokenization disparity while maintaining competitive downstream performance, and suggest that Indic-specialized adaptive segmentation can reduce tokenization disparity while maintaining competitive downstream performance.
COILD, an Indic-centric parallel corpus comprising over 1.16 million human-translated and human-verified sentence pairs, covers 20 Indian language pairs across the Indo-Aryan, Dravidian, Tibeto-Burman, and Austro-Asiatic language families, and provides a valuable training and evaluation resource for advancing multiling...
Kshetrimayum Boynao Singh, Nitin Kumar Mishra, P. Dutta et al.· arXiv.org· 0 citations
Safety alignment for Large Language Models is heavily focused on English with limited multilingual representation, particularly damaging low-resource language performance. Related open training datasets are often limited in size or do not follow region-localized harmful topics to broaden the scope of alignment beyond c...
Max Neuwinger, H. Yukhymenko, Imanol Schlag· 0 citations
Low-resource Indian languages continue to face significant challenges in neural machine translation due to the scarcity of high-quality parallel corpora and the diversity of scripts and linguistic characteristics. This paper presents MTG-LIRA’s submission to the WMT 2026 Low-Resource Indic Language Translation Shared T...
Vedarth Hede, Neha Gupta, H. Bapat et al.· Proceedings of the Eleventh...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.