Deep Dive: Comparing Trillion-Scale Open-Weight Mixture-of-Experts Models
A comprehensive comparison of three leading open-weight Mixture-of-Experts models: Kimi K3, DeepSeek V4 Pro, and GLM-5.2. It analyzes their benchmark scores, license differences, and infrastructure serving costs.
Impact: High
Why it matters
Developers can choose the ideal open-weight model based on task cost, serving speed, and deployment feasibility.
TL;DR
- 01DeepSeek V4 Pro is the clear cost leader at $0.18 per million blended tokens.
- 02Kimi K3 offers top-tier reasoning capabilities (ranking #3 overall) but remains API-only until weights release on July 27, 2026.
- 03GLM-5.2 provides the fastest throughput at 168 tokens/second and can be self-hosted with 8x H200 GPUs.
Key facts
- Kimi K3 Parameter Count
- 2.8T
- DeepSeek V4 Pro Blended Cost
- $0.18 per MTok
- GLM-5.2 Inference Speed
- 168 t/s
- Context Window for All Three
- 1M tokens
- Kimi K3 Weight Release Date
- July 27, 2026
Performance and Architecture
All three models utilize sparse Mixture-of-Experts (MoE) architectures with massive 1M-token context windows:
- Kimi K3: 2.8T parameters (activating 16/896 experts). It supports text, vision, and video.
- DeepSeek V4 Pro: 1.6T parameters (49B active, 384 routed experts).
- GLM-5.2: 744B parameters (~40B active).
On the Artificial Analysis Intelligence Index, Kimi K3 scores 57 (#3 overall, comparable to Claude 4.8 Opus), GLM-5.2 scores 51, and DeepSeek V4 Pro scores 44 (rising with Max reasoning).
API Pricing and Cost Efficiency
DeepSeek V4 Pro dominates on economic metrics. Blended rates (7:2:1 cache/input/output ratio) evaluated by Artificial Analysis put DeepSeek V4 Pro at $0.18 per million tokens, compared to $0.90 for GLM-5.2 and $2.31 for Kimi K3. At list rates, one dollar buys roughly 1.15M output tokens from DeepSeek V4 Pro, but only 67k from Kimi K3.
Speed and Self-Hosting Trade-Offs
GLM-5.2 is the speed champion, delivering approximately 168 tokens/second, compared to around 62 tokens/second for both Kimi K3 and DeepSeek V4 Pro.
For self-hosting:
- GLM-5.2: Requires over 1TB VRAM, achievable with an 8x H200 GPU cluster running FP8.
- DeepSeek V4 Pro: Requires larger clusters due to its 1.6T scale.
- Kimi K3: Demands at least 64 accelerators, making local serving impractical for smaller teams.
✓ When to use
- When evaluating open-source LLM options for high-throughput, long-context applications.
- When planning self-hosted infrastructure budgets for large-scale enterprise deployments.
✕ When NOT to use
- If you need low-latency, lightweight edge inference models that run comfortably on single consumer GPUs.
What to do today
- Evaluate DeepSeek V4 Pro for highly repetitive production agent pipelines to cut costs.
- Monitor Hugging Face on July 27, 2026, for the open-weight release of Kimi K3.
What the community says
“"race to the bottom" is the negative framing of "competitive market prices". So a company or union might say "this is a race to the bottom" when someone new enters their market...”
“I'm starting to think It might be a good idea to stay with American models, by virtue of them feeling so threatened they give out more and more free perks.”
Sources