#artificial intelligence #machine learning #natural language processing Preprint Open access

Nemotron 3.5 Super and Lightning: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Models for Agentic AI

NVIDIA : Aastha Jhunjhunwala Aayushi Ashokkumar Srivastava Abhibha Gupta Abhinav Garg Abhinav Khattar Achsah Wildish Adam Rajfer Adi Renduchintala Adil Asif Adriana Flores Miranda Aileen Zaman Aishwarya Chhabra Akanksha Shukla Akhiad Bercovich Alejandra Rico Aleksa Gordic Aleksander Ficek Alex Gronskiy Alex Steiner Alexander Bukharin Alexandre Milesi Ali Roshan Ghias Ali Shehper Ali Taghibakhshi Alice Gatti Alisa Liu Alok Kumar Alok Tripathy Amala Sanjay Deshmukh Aman Miryala Aman Shanbhag Ameya Sunil Mahabaleshwarkar Ameya Velingker Amir Klein Amir Samani Amit Zuker Amnon Geifman Anahita Bhiwandiwalla Ananth Subramaniam Andre Manoel Andrea Santilli Andrew Tao Andrii Skliar Anil Thomas Anish Mahishi Anna Shors Anna Warno Anuj Doshi Anusha Kamath Anwith Kiran Ao Tang Arham Mehta Arnab Dhara Arnav Komaragiri Arnav Kundu Arti Jain Arun Rao Arun Venkatesan Arushi Goel Asha Anoosheh Ashish Vaswani Ashwath Aithal Asif Ahamed Asma Kuriparambil Thekkumpate Aswin Visva Atefeh Sohrabizadeh Avinash Vem Ayush Dattagupta Benedikt Schifferer Benjamin Chislett Bernard Nguyen Besmira Nushi Bilal Kartal Bita Darvish Rouhani Bobby Chen Boris Ginsburg Boxin Wang Brandon Norick Branislav Kisacanin Brendan Johnson Brian Dang Brian Yu Bryan Catanzaro Byung-Kwan Lee Carlo del Mundo Chankyu Lee Chantal Hwang Charles Wang Charlie Truong Chen Cui Cheng Wang Cheng-Ping Hsieh Chenhan Yu Chenjie Luo Chintan Patel Chris Alexiuk Chris Johnson Chris Wing Christian Munley Coco Li Collin McCarthy Cynthia Gao Dan Su Dan Zhao Dane Corneil Danial Mohseni Taheri Daniel Afrimi Daniel Fatade Daniel Lo Daniel Machlab Daniel Serebrenik Daniele Molinari Daniil Sorokin Daria Gitman Daria Levy David Mosallanezhad David Yu Deepak Narayanan Dev Churiwala Devin O'Kelly Dhruv Nathawani Di Allan Wu Dima Rekesh Dina Yared Dipendra Misra Divyanshu Kakwani Dmitry Belenko Dominic Manno Dongyang Fan Dor Tsur Duncan Riach Dustin VanStee Dylan Lu Edgar Minasyan Ehsan Hosseini-Asl Eileen Long Elena Lantz Ellie Evans Elliott Ning Elvis Chen Emanuel Scoullos Eric Chung Eric Harper Eric Pham-Hung Eric Tramel Eric Yang Erik Arakelyan Evelina Bakhturina Ewa Dobrowolska Fadi Saady Farzan Memarian Fay Wang Fei Jia Felipe Soares Felipe Vieira Frujeri Feng Chen Fengguang Lin Ferenc Galko Frankie Siino Frederic Lafrance Fuxiao Liu Gantavya Bhatt Garvit Kulshreshtha George Armstrong Giulio Lovisotto Gomathy Venkata Krishnan Gosia Lazuka Grace Lam Greg Heinrich Greg Pauloski Grzegorz Chlebus Guilin Liu Guo Chen Guoming Zhang Guyue Huang Haifeng Qian Hanrong Ye Hao Gao Hao Zhang Haoqin Tu Haran Kumar Shiv Kumar Haribhau Hud Harris Nover Harrison Saturley-Hall Harshil Kotamreddy Hemil Desai Henry Estela Hexin Wang Himanshu Singh Hui Wang Hung-Yueh Chiang Igor Gitman Ilia Karmanov Ilya Loshchilov Ingo Guehring Itay Neeman Ivan Moshkov Jacky Kwok Jakub Slowikowski James Shen Jane Polak Scowcroft Jared Casper Jatin Mitra Jeff Willette Jenna Diamond Jenny Chen Jesse Oliver Jessica Medina Jiacheng Xu Jialin Song Jian Hu Jian Tu Jian Zhang Jiantao Jiao Jiaqi Zeng Jie-Fang Zhang Jimmy Zhang Jin Xu Jingyu Xin Jinhang Choi Joey Conway Joey Guman John Kamalu John Liu John Roberts Johnny Greco Jonah Alben Jonas Yang Jonathan Cohen Jonathan Li Jonathan Raiman Joseph Jennings Juhi Mittal Julia Gusak Julien Veron Vialard June Yang Junkeun Yi Jupinder Parmar Ka Shun Shum Kaiwen Shi Kajal Jain Kanishk Singla Karan Sapra Kari Briski Kateryna Chumachenko Katherine Huang Katie Lyons Keith Wyss Keshav Santhanam Keval Morabia Kevin Shih Kezhi Kong Khanh Nguyen Khushi Bhardwaj Kiran Praveen Konstantinos Krommydas Krishna Puvvada Krzysztof Pawelec Kushal Shah Li Ding Lily Lee Linda Liu Linglin Jing Luis Vega Maarten Van Segbroeck Mahan Fathi Mahdi Nazemi Maka Dong Makesh Narsimhan Sreedhar Maor Ashkenazi Marc Cuevas Marcin Chochowski Marco Pavone Markus Kliegl Marta Stepniewska-Dziubinska Martina Gonzalez Vilas Martyna Patelka Matan Rusanovsky Matthieu Le Matty Machczynski Matvei Novikov Mauricio Ferrato Meenakshi Sharma Mehrzad Samadi Melissa Corpuz Melissa Saelzer Meng Xin Mengru Wang Mengxi Wu Meriem Boubdir Michael Boone Michael Callahan Michael Elad Michael Lightstone Michal Bien Michal Zawalski Miguel Martinez Mikail Khona Mike Chrzanowski Mike Ranzinger Mike Wojcikiewicz Min-Hung Chen Ming Kang Mingjie Liu Mingyuan Ma Misha Nekorystnov MJ Mikulski Mohammad Qazim Bhat Mohammad Shoeybi Morgan Purcell Mostofa Patwary Nabin Mulepati Nancy Agarwal Narimane Hennouni Nathan Sala Nave Assaf Nayeon Lee Neal Vaidya Negar Habibi Netanel Haber Nick Gupta Nick Reamaroon Nickson Quak Nidhi Bhatia Nik Spirin Niket Agarwal Nikolai Ludwig Nima Tajbakhsh Nithin Sunny Oleksii Hrinchuk Oleksii Kuchaiev Olivier Delalleau Omer Ullman Argov Pablo Ribalta Padmavathy Subramanian Parth Mannan Pavlo Molchanov Pawel Morkisz Peter Jin Petr Lapukhov Petro Junior Milan Philip Monk Pinky Xu Piotr Januszewski Piotr Laszkiewicz Po-Han Huang Praateek Mahajan Pradeep Thalasta Prakash Gurumurthy Pranav Prashant Thombre Prasoon Varshney Pratyusha Maiti Preeti Chadha Pritam Biswas Pritam Gundecha Przemek Strzelczyk Puhui Meng Pulkit Kumar Rabeeh Karimi Mahabadi Rachel Oberman Rajpreet Thethy Ramani Akkati Rameshwar Shivbhakta Ran Zilberstein Randy Gelhausen Rasoul Shafipour Raviraj Joshi Ray Macalisang Rayen Tian Reka Kovacs Renjie Pi Resha Shetty Ria C Rima Shahbazyan Rishabh Garg Rita Fernandes Neves Ritika Borkar Ritu Gala Riya Talwar Riyad Islam Robert Clark Robert Kirby Roger Waleffe Rohan Joshi Rohan Pillai Rohit Jena Rohit Watve Roi Garnett Roi Koren Rongguang Ye Russell J. Hewett Ryan Prenger Ryan Stewart Ryota Egashira Sadegh Mahdavi Sai Nishanth Padala Salika Dave Sandhya Rani Narravula Sandip Bhaskar Sangkug Lym Sanjeev Satheesh Sarah Amirslani Saran Vikas Murari Saurabh Mishra Saurabh Srivastava Saurav Muralidharan Sean Narenthiran Sean Wang Seokju Cho Seong Hee Lee Seonjin Na Seonmyeong Bak Seph Mard Serge Panev Sergei Kolchenko Sergio Perez Seth Henneman Seth Poulos Shahar Mor Shantanu Acharya Shaokun Zhang Shaona Ghosh Sharath Turuvekere Sreenivas Shaun Kotek Shaya Gharghabi Sheng-Chieh Lin Shengliang Xu Shi Chen Shihao Wang Shilpa Amireddy Shiqing Fan Shiyang Chen Shrimai Prabhumoye Shriram Venugopal Shuang Yu Shubham Gandhi Shubham Pachori Shubham Toshniwal Shuoyang Ding Shwetha Krishnamurthy Simeng Sun Sivakumar Arayandi Thottakara Somshubra Majumdar Soumye Singhal Sridurga Krithivasan Sriharsha Niverty Stephen Ge Sugam Dipak Devare Sukrit Rao Sumeet Kumar Barua Sungsoo Ha Surya Nistala Suseella Panguluri Sweta Priyadarshi Syeda Nahida Akter Talor Abramovich Tanay Varshney Tandeep Singh Chandehoke Teo Ene Terry Kong Thanh Do Thinh Le Tian Zhang Tijmen Blankevoort Tim Romanski Tiyasa Mitra Tom Balough Tom Huang Tomasz Grzegorzek Tomer Asida Tomer Bar Natan Traian Rebedea Trenton Starkey Tugrul Konuk Tuomas Rintamaki Twinkle Vashishth Tyler Kniess Tyler Poon Udi Karpas Ushnish De Utkarsh Vaidya Vahid Noroozi Vanshil Atul Shah Venkat Srinivasan Venkata Karri Venmugil Elango Vibhor Agrawal Vibhu Jawa Victor Cui Victoria Edmonds Vijay Korthikanti Vikas Mehta Vikranth Jeyakumar Vinay Rao Vineeth Kalluru Srinivas Vitaly Kurin Vitaly Lavrukhin Wahidur Rahman Wanli Jiang Wasi Uddin Ahmad Wataru Ishihara Wei Du Wei Ping Wei-Ming Chen Weiheng Chai Weijia Chen Wendy Quan Wenliang Dai Wenyan Cong William Zhang Wojciech Prazuch Xiaowei Ren Xin Yu Xiwen Yu Yan Wang Yang Chen Yang Yu Yangyi Chen Yaniv Galron Yash Jain Yashaswi Karnati Ye Yu Yejin Choi Yemi Ibrahim Yev Meyer Yi Dong Yi-Fu Wu Yian Zhang Yifan Zhang Ying Lin Yiyang Zhou Yoav Miron Yogendra Pandey Yonatan Geifman Yonggan Fu Yoshi Suhara Youngeun Kwon Yu-Cheng Chou Yu-Chiang Frank Wang Yuanguo Kuang Yuekai Zhang Yugi Guvvala Yuki Huang Yunheng Zou Yunsheng Liu Zhenming Wang Zhiding Yu Zhiqi Li Zhiyu Cheng Zhongbo Zhu Zhuolin Yang Zichao Yue Zihan Liu Zijie Yan Zirui Liu Zoe Faul Zuhair Ahmed

Oct 2026

Artificial Intelligence Machine Learning Natural Language Processing

Abstract

We introduce Nemotron 3.5 Super and Nemotron 3.5 Lightning, open and efficient Mixture-of-Experts Hybrid Mamba-Transformer models designed for reasoning, long-context understanding, and agentic workloads. Super is a 120B-parameter, 12B-active multimodal model, while Lightning is a smaller 30B-parameter, 3B-active text model; both designed for efficient deployment. Both models combine large-scale distillation, long-context training, and native speculative decoding, while Super further incorporates Indic continued pretraining and agentic software-engineering mid-training. We develop a comprehensive post-training pipeline spanning supervised fine-tuning and reinforcement learning (RLVR and RLHF), complemented in Super by specialized teacher training and multi- teacher on-policy distillation (MOPD). This transforms Super from a text-only base model into a multimodal agent capable of reasoning over text, images, video, and interactive tool environments. We further explore test-time scaling through parallel agent execution and self-verification. Across training and deployment, we jointly optimize capability and efficiency. Indic continued pretraining improves Indian-language benchmarks by up to 9.82 percentage points, while agentic mid-training strengthens software-engineering capabilities and robustness across agent harnesses. The post-trained Super model achieves 51.78% on SWE-bench Pro, surpassing Nemotron 3 Ultra despite using approximately 4.6x fewer active parameters. Post-training MTP boosting improves speculative- decoding acceptance length by up to 21.8% under probabilistic draft sampling, while NVFP4 quantization achieves 99.62% median accuracy recovery relative to BF16. Inference optimizations enable up to 4,856 output tokens/s/GPU on GB200 while sustaining at least 50 tokens/s/user.

DOI arXiv

View source