NixOS Module Enables Reproducible AI Workloads on NVIDIA DGX Spark
A new NixOS module and USB image collection simplifies deploying reproducible AI environments on NVIDIA DGX Spark and Asus Ascent GX10. It includes pre-configured devshells for fine-tuning, vLLM inference, and telemetry without manual CUDA setup.
Impact: Medium
Why it matters
You can now run fully reproducible, container-free deep learning and inference pipelines natively on NVIDIA edge hardware using Nix.
TL;DR
- 01Enables reproducible local ML development without the fragmentation of traditional virtual environments or heavy Docker images.
- 02Provides pre-built CUDA packages via Flox binary cache, drastically reducing deployment compile times.
- 03Optimizes the kernel configuration by 82% over generic builds for hardware stability.
Key facts
- DGX Dashboard Port
- 11000
- NixOS Kernel Option
- Specialized NVIDIA Kernel (default) or Standard 6.17
- Supported hardware
- NVIDIA DGX Spark, Asus Ascent GX10
- Kernel Config Size Reduction
- 82%
Quick Installation and Configuration
To deploy this configuration on an NVIDIA DGX Spark system, you can initialize a template using flakes. This sets up a complete layout including flake.nix and configuration.nix files.
nix flake init -t github:graham33/nixos-dgx-spark#dgx-sparkCustom NVIDIA Kernel
The module defaults to using a specialized NVIDIA-optimized kernel build. It is generated from NVIDIA's Debian annotations, keeping configuration diffs minimal:
hardware.dgx-spark.enable = true;
# useNvidiaKernel = true; (default)If you encounter networking issues with standard kernels, sticking to this customized version ensures that both Ethernet and full GPU support function correctly. The module also activates fwupd, allowing firmware updates published by NVIDIA directly through the Linux Vendor Firmware Service (LVFS).
Running Non-Containerized Workloads
The repository provides devshells configured to run PyTorch fine-tuning, vLLM inference server (with models like Qwen2.5-Math-1.5B-Instruct), and TensorRT FP4 quantization without the overhead of Docker containers. For non-NixOS environments (e.g., standard Ubuntu-based DGX OS), you can install Nix and use the official binary cache to avoid compiling CUDA dependencies:
extra-substituters = https://cache.flox.devTry it in 2 minutes
nix run github:nix-community/nixos-anywhere -- --flake github:graham33/nixos-dgx-spark#dgx-spark root@<ip>bash
✓ When to use
- Deploying NixOS or declarative Nix environments on NVIDIA DGX Spark or Asus Ascent GX10 hardware.
- Running reproducible local deep learning pipelines without the overhead of heavy Docker containers.
✕ When NOT to use
- If you do not own or manage specialized DGX Spark or Asus Ascent GX10 hardware.
- If you rely entirely on cloud-based serverless API inference endpoints.
What to do today
- Clone the nixos-dgx-spark repository or import it as a flake in your NixOS configurations.
- Configure the Flox binary cache in your nix.conf to avoid compiling heavy CUDA libraries.
What the community says
“Been running this on a few Asus GX10 machines with k3s on top, it’s been great. I’m running the new deepseek. Thank you for your work!”
“This has been amazingly helpful for managing my DGX Spark! Thank you for all your time and effort into this project!”
Sources