NVIDIA and Hugging Face Release NeMo Automodel for Scalable Diffusers Fine-Tuning
NVIDIA and Hugging Face have launched NeMo Automodel, a PyTorch DTensor-native library. It enables zero-checkpoint-conversion fine-tuning of multi-billion parameter diffusion models like FLUX.1-dev and HunyuanVideo.
Impact: Medium
Why it matters
Developers can now fine-tune massive open-source video and image models on custom datasets at any scale by editing simple YAML configurations instead of rewriting custom training scripts.
TL;DR
- 01NeMo Automodel integrates Hugging Face Hub Diffusers models directly with distributed PyTorch DTensor training.
- 02Supports advanced FSDP2, tensor, context, and pipeline parallelisms via declarative configuration.
- 03Eliminated manual checkpoint format conversion steps for video and image model pipelines.
Key facts
- Open Source License
- Apache 2.0
- Recommended Docker Image
- nvcr.io/nvidia/nemo-automodel:26.06
- Core Training Framework
- PyTorch DTensor
Zero Checkpoint Conversion and Native Integration
NeMo Automodel is an open-source PyTorch DTensor-native training library published under the Apache 2.0 license. Pretrained weights are loaded directly from the Hugging Face Hub using the familiar pretrained_model_name_or_path parameter, and output checkpoints load directly back into standard Hugging Face pipelines. This preserves downstream compatibility with tools like quantization, compilation, and custom samplers.
Scaling via Declarative YAML Configurations
With Automodel, parallelism is a configuration choice rather than a code rewrite. Switch between Fully Sharded Data Parallel 2 (FSDP2), tensor, context, and pipeline parallelisms by declaring parameters in a YAML config.
CLI Walkthrough for FLUX.1-dev Fine-Tuning
To fine-tune a model, stream your dataset and preprocess the VAE latents using multiple GPUs:
python -m tools.diffusion.preprocessing_multiprocess image \
--dataset_name multimodalart/1920-raider-waite-tarot-public-domain \
--model_name black-forest-labs/FLUX.1-devRun the fine-tuning script with FSDP2 enabled using the checked-in recipes:
uv run --locked --no-default-groups --extra diffusion \
examples/diffusion/finetune/finetune.py \
-c examples/diffusion/finetune/flux_t2i_flow.yaml \
--data.dataloader.cache_dir /cache/flux_tarot \
--data.dataloader.base_resolution '[384,640]' \
--lr_scheduler.lr_decay_style constant \
--checkpoint.checkpoint_dir /tmp/flux_tarot/checkpoints/fullOnce training completes, run inference directly from the resulting checkpoint directory:
uv run --locked --no-default-groups --extra diffusion \
python examples/diffusion/generate/generate.py \
-c examples/diffusion/generate/configs/generate_flux.yaml \
--model.checkpoint /tmp/flux_tarot/checkpoints/full/epoch_66_step_199 \
--inference.prompts '["a trtcrd of an astronaut tending a rose garden on Mars"]' \
--output.output_dir /tmp/flux_tarot/generations/full/step_200Try it in 2 minutes
pip3 install nemo-automodelbash
✓ When to use
- When fine-tuning multi-billion parameter text-to-image or text-to-video models across single or multi-GPU systems.
✕ When NOT to use
- If you are fine-tuning non-diffusion architectures, as this tool is tailored specifically to flow-matching diffusion models.
What to do today
- Pull the pre-built Docker image `nvcr.io/nvidia/nemo-automodel:26.06` to bypass building CUDA dependencies.
- Run the provided tarot dataset preprocessing script on your local GPU to cache VAE latents before fine-tuning.
Sources