Reflection Releases Beam, an Open 501B Mixture-of-Experts Coding Model
Reflection has published Beam, an open-source 501B Mixture-of-Experts model under Apache 2.0 with a 1M token context window. Activating 23B parameters per token, it offers day-0 support across Hugging Face Transformers, vLLM, and SGLang.
Why it matters
Teams can now host and fine-tune an unrestricted, frontier-grade long-context coding model on their own infrastructure without SaaS API locks.
TL;DR
- 01Beam brings an open Apache 2.0 alternative to proprietary 1M context coding models.
- 02Hybrid attention cuts KV cache overhead by roughly 83% (down to 36 KiB per token).
- 03NVFP4 compression allows running 501B weights on 4x NVIDIA B200 instances with full 1M context headroom.
Key facts
- Total Parameters
- 501B
- Active Parameters
- 23B per token
- Context Window
- 1,000,000 tokens
- License
- Apache 2.0
- Architecture
- 52 layers, 256 experts, 8 routed per token
- KV Cache Size
- 36 KiB per token (hybrid attention)
Architecture and Hybrid Attention
Beam features a 501B-parameter Mixture-of-Experts (MoE) structure where only 23B parameters are active per token. The network spans 52 layers with a hidden dimension of 5,120. The first 3 layers are dense SwiGLU MLPs, followed by 49 MoE layers containing 256 experts each (holding 98.4% of total weights). Routing follows a DeepSeek-V3 group-limited approach, picking the top 8 experts across the 4 best groups.
To support its 1,000,000-token context without exploding memory, Beam combines 43 sliding-window layers (restricted to a 512-token window) with 9 full-sequence global attention layers. Sliding-window layers use standard RoPE, while global layers use scalable-softmax query scaling without positional embeddings. This drops KV-cache growth from 208 KiB to 36 KiB per token.
Quantization and Hardware Sizing
Beam is released in three precision profiles:
- BF16: ~1 TB disk/VRAM footprint, requiring distributed multi-node clusters.
- NVFP4: ~280 GB footprint designed natively for NVIDIA Blackwell GPUs, fitting on a node of 4x B200 GPUs (720 GB VRAM) with space for concurrent agent execution.
- FP8: Optimized for Hopper (H100/H200) and Ampere architectures.
Day-0 Ecosystem Integration
Inference is supported out of the box via transformers, vLLM, and SGLang. Reflection also provides native recipes for Supervised Fine-Tuning (SFT) and Reinforcement Learning via Miles. The model incorporates a multi-tier reasoning parameter (preserve_thinking) spanning five levels: low, medium (default), high, xhigh, and max.
Try it in 2 minutes
from transformers import pipeline
pipe = pipeline("text-generation", model="reflection/Beam-1.0-501B-A23B", device_map="auto")
messages = [{"role": "user", "content": "Write an optimized quicksort in Rust."}]
output = pipe(messages, max_new_tokens=2000)python
✓ When to use
- Self-hosting an enterprise-grade agent orchestration model with long context in private infrastructure.
- Fine-tuning custom coding engines on proprietary repositories under permissive Apache 2.0 licensing.
✕ When NOT to use
- Small developer setups lacking high-memory enterprise GPUs (requires at least FP8/NVFP4 on enterprise nodes).
- Low-latency real-time chat interactions where smaller 8B–14B models suffice with minimal cold-start times.
What to do today
- Test Beam locally or on serverless providers using the official Hugging Face Transformers pipeline.
- Deploy the NVFP4 or FP8 weights in vLLM or SGLang clusters for production agent evaluations.
- Apply for the $10,000 Hugging Face Hub credits pool by publishing custom quantizations or agent benchmarks.