PyTorch 2.15 Adds FP8 and Native Metal Kernels for Apple Silicon MPS
PyTorch 2.15 overhauls the Metal Performance Shaders (MPS) backend on Apple Silicon with FP8 tensor support and native grouped matrix multiplication. Mixture-of-experts model decoding runs up to 6.5x faster, and video VAE decoding memory drops from 270 GiB to 8 GiB.
Why it matters
You can now run larger local models in half the memory on Macs while accelerating MoE and linear layer decoding throughput.
TL;DR
- 01FP8 tensors (float8_e4m3fn) halve memory usage for weights on Apple Silicon Macs.
- 02Mixture-of-experts decoding achieves 5x to 6.5x speedups via GPU-resident grouped matrix multiplication.
- 03System requirements are updated to macOS 15 or later and Python 3.11 or later.
Key facts
- Qwen3.6-35B-A3B decode speed
- 61.3 tokens/s (up from 9.4 on M3 Ultra)
- LFM2.5-8B-A1B decode speed
- 123.6 tokens/s (up from 23.6 on M3 Ultra)
- Wan2.2 video VAE peak memory
- 8 GiB (down from 270 GiB)
- Minimum OS requirement
- macOS 15+
FP8 Tensors and Scaled Matrix Multiplication
PyTorch 2.15 introduces torch.float8_e4m3fn support directly to Apple Silicon via MPS. FP8 tensors consume one byte per weight value—cutting storage in half compared to FP16 or BF16. Matrix operations run via F.scaled_mm, which supports both tensorwise and rowwise scaling recipes.
MoE Decoding Gains Up to 6.5x
During LLM token decoding, inputs enter as single-row vectors. PyTorch 2.15 wires optimized GEMV kernels directly into F.linear, removing previous MPSGraph overhead. For Mixture-of-Experts architectures, F.grouped_mm replaces CPU-GPU synchronization with GPU-native routing. Benchmark results on an M3 Ultra with torch.compile show:
Qwen3.6-35B-A3B: increased from 9.4 to 61.3 tokens/s (6.5x faster).LFM2.5-8B-A1B: increased from 23.6 to 123.6 tokens/s (5.2x faster).- Vocabulary softmax operations decode roughly 8x faster.
Video Memory and Linear Algebra Fixes
Runaway driver memory allocation in tiled video generation has been contained. Decoding video with Wan2.2 tiled VAE previously consumed up to 270 GiB of driver memory; it now caps at 8 GiB. Complex linear algebra operations (complex64 LU factorization, LU solve, triangular solve) and batched small-matrix (up to 4x4) inverses now run in specialized Metal kernels. Note that PyTorch 2.15 bumps minimum requirements to macOS 15 and Python 3.11.
Try it in 2 minutes
uv pip install torch==2.15.0 --index-url https://download.pytorch.org/whl/test/cpubash
✓ When to use
- Running local LLM, MoE, and ComfyUI generation pipelines on Apple Silicon hardware.
- Constrained unified memory environments needing FP8 weight quantization without quality collapse.
✕ When NOT to use
- Machines running macOS 14 or older, which are no longer supported by PyTorch 2.15 MPS.
- Production inference pipelines hosted exclusively on NVIDIA CUDA cloud clusters.
What to do today
- Upgrade local Apple Silicon test environments to macOS 15 and Python 3.11+.
- Install the PyTorch 2.15 release candidate via uv or pip to benchmark local MoE inference.
- Convert local model weights to torch.float8_e4m3fn to reduce unified memory footprint.