NVIDIA has published Nemotron 3 Ultra, an open mixture-of-experts hybrid Mamba-Transformer model with 550 billion total parameters, 55 billion active parameters, and support for up to a 1 million-token context window.
NVIDIA published details for Nemotron 3 Ultra on June 4, describing a new open model aimed at long-context and agentic reasoning workloads.
NVIDIA’s release page describes Nemotron 3 Ultra as a mixture-of-experts model with 550 billion total parameters and 55 billion active parameters. The accompanying NVIDIA Research technical report uses the same configuration and identifies the model as a hybrid Mamba-Transformer system, rather than a standard Transformer-only architecture.
According to the technical report, Nemotron 3 Ultra was pretrained on 20 trillion tokens. NVIDIA Research says the model was then extended to support a context length of up to 1 million tokens and post-trained using supervised fine-tuning, reinforcement learning, and a method referred to in the report as MOPD.
The Hugging Face model card for NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4 also lists 550 billion total parameters, 55 billion active parameters, and a 1 million-token context window. The model card identifies the checkpoint as an NVFP4 version and lists the license as OpenMDW-1.1.
NVIDIA’s release page says Nemotron 3 Ultra reaches benchmark parity with state-of-the-art open large language models. The company also claims higher inference throughput compared with GLM, Kimi, and Qwen models, according to the same release page.
Those performance claims are presented by NVIDIA in its own materials, and the available excerpts do not provide enough detail to independently assess the benchmark methodology, hardware setup, serving configuration, or exact comparison points. The technical report is the primary source for the model’s architecture, training, and post-training description.
NVIDIA Research frames Nemotron 3 Ultra as a model for “agentic reasoning,” a term commonly used for systems that must plan, use tools, process long instructions, or maintain state across extended interactions. The reported 1 million-token context window is central to that positioning, as it could allow the model to process very large codebases, document sets, or multi-step task histories in a single context.
The model’s mixture-of-experts design is also relevant to deployment. In MoE systems, only a subset of parameters is typically active for a given token. NVIDIA lists Nemotron 3 Ultra as having 55 billion active parameters out of 550 billion total, which indicates that inference uses only part of the model at each step. NVIDIA’s release page links this design to its throughput claims against other open models.
NVIDIA says in the technical report that Nemotron 3 Ultra has been open-sourced on Hugging Face. The Hugging Face listing for the NVFP4 checkpoint shows a June 4, 2026 release date and provides the public model entry under NVIDIA’s account.
The NVFP4 checkpoint suggests a focus on lower-precision deployment, though the source excerpts only identify the format and do not provide full deployment requirements. Users evaluating the model would still need to review the model card, license, and technical report for permitted uses, hardware assumptions, and implementation details.
Nemotron 3 Ultra adds another large open model to the high-parameter MoE category, with NVIDIA emphasizing long context, inference efficiency, and reasoning-oriented post-training. The strongest source-backed facts are the model’s 550B-total and 55B-active parameter counts, 20T-token pretraining scale, 1M-token context length, hybrid Mamba-Transformer design, and Hugging Face availability under the OpenMDW-1.1 license.
NVIDIA published details for Nemotron 3 Ultra on June 4, describing a new open model aimed at long context and agentic reasoning workloads.
A 550B parameter model with 55B active parameters NVIDIA’s release page describes Nemotron 3 Ultra as a mixture of experts model with 550 billion total parameters and 55 billion active parameters.
The accompanying NVIDIA Research technical report uses the same configuration and identifies the model as a hybrid Mamba Transformer system, rather than a standard Transformer only architecture.
Continue reading
