Alibaba Open-Sources Qwen3.8 2.4-Trillion Parameter Mixture of Experts Model
Alibaba released open weights for Qwen3.8-2.4T-A95B with 2.4 trillion parameters and 95B active per token. Featuring a hybrid linear/full attention architecture and configurable reasoning depth, it serves at 4K tokens/sec/GPU on NVIDIA GB300 systems.
Impact: High
Why it matters
You can now run frontier-scale 1M token context reasoning agent workflows on self-hosted infrastructure using standard vLLM or SGLang engines.
TL;DR
- 01Qwen3.8-2.4T-A95B provides 1M context with 95B active parameters per token.
- 02Built-in reasoning modes (low/high/xhigh) let you toggle execution latency vs response quality per API request.
- 03Day-0 deployment is fully supported in vLLM, SGLang, and NVIDIA NIM.
Key facts
- Total Parameters
- 2.4 Trillion
- Active Parameters
- 95 Billion per token
- Context Window
- 1,000,000 tokens
- Max Output Length
- 128,000 tokens
- Throughput (GB300 NVL72)
- >4,000 tokens/sec/GPU (self-reported)
Hybrid Attention Architecture
Qwen3.8-2.4T-A95B combines standard full-attention layers with gated linear-attention layers. In linear layers, the unbounded KV cache is converted into a fixed-size recurrent state, keeping memory overhead manageable across its 1M context window.
Fine-Grained MoE Routing
With 2.4T total parameters, only 95B parameters are activated per token. A learned router dynamically assigns tokens to specialized small experts, keeping compute costs aligned with active parameter counts rather than total model capacity.
Production Deployment Stack
Developers can deploy the model on Day-0 using standard open inference runtimes:
vLLMandSGLangdistributed serving recipes- NVIDIA NIM model-free containers on NGC
- Fine-tuning via
NVIDIA NeMo AutoModelusing Hugging Face checkpoints directly
Try it in 2 minutes
docker run --gpus all -p 8000:8000 nvcr.io/nim/alibaba/qwen3.8-2.4t-a95b:latestbash
✓ When to use
- Use when building complex multi-step reasoning coding agents requiring large context windows.
- Deploy on self-hosted high-density enterprise GPU clusters requiring frontier model capabilities.
✕ When NOT to use
- Do not deployment on single consumer GPUs without distributed multi-node hardware.
- Avoid using for lightweight, ultra-low latency real-time edge processing.
What to do today
- Pull model weights from Hugging Face or ModelScope for self-hosted LLM clusters.
- Configure inference runtimes with vLLM or SGLang targeting FP8 precision.
- Test low vs xhigh reasoning control flags on agent workflow tasks.
Sources