NVIDIA Releases Nemotron Labs 3 Puzzle 75B Latent Mixture of Experts Model
NVIDIA published Nemotron-Labs-3-Puzzle-75B-A9B-NVFP4, a 75-billion parameter Latent Mixture of Experts model quantized using NVFP4, built on the Nemotron-3-Super-120B base.
Why it matters
This release provides an implementation of a Latent Mixture of Experts architecture utilizing NVFP4 quantization for specific model layers.
TL;DR
- 01Uses Latent Mixture of Experts architecture.
- 02Implements NVFP4 quantization on specific projection layers.
- 03Supports 7 languages including Japanese and Chinese.
Custom NemotronHPuzzle Architecture
NVIDIA's new release introduces NemotronHPuzzleForCausalLM under the openmdw-1.1 license. This model is a Latent Mixture of Experts (MoE) with support for multi-token prediction (mtp). It supports native configuration via Hugging Face's AutoConfig and AutoModelForCausalLM loaders.
NVFP4 Quantization
A core feature of the Puzzle-75B-A9B model is its NVFP4 quantization configuration. The weight matrices across various mixer projection layers—including in_proj, out_proj, and MoE shared experts down_proj and up_proj—are quantized according to defined group configurations.
Multilingual Text Generation
The model supports English, French, Spanish, Italian, German, Japanese, and Chinese, trained on the nvidia/nemotron-post-training-v3 and nvidia/nemotron-pre-training-datasets corpora.