Liquid AI Accelerates Local Vision-Language Models with LFM2.5-VL-DSpark Speculative Decoding
Liquid AI released an experimental DSpark draft model for its LFM2.5-VL-3B vision-language model, adding speculative decoding to local and server runtimes. It delivers up to 3.13x faster decoding on edge devices without altering output accuracy.
Impact: High
Why it matters
Accelerate local multimodal inference up to 2.62x end-to-end on Apple Silicon and server GPUs using official builds of llama.cpp, MLX, and SGLang.
TL;DR
- 01Adds speculative decoding to the 3B parameter LFM2.5-VL with an 8.9% memory overhead (280M parameters).
- 02Provides 1.56x to 2.62x end-to-end latency reduction on Apple Silicon and up to 2.27x on H100 GPUs.
- 03Guarantees exact output equivalence without quality degradation during greedy sampling.
- 04Ships with ready-to-run configurations for llama.cpp, MLX-VLM, and SGLang.
Key facts
- Drafter Parameter Count
- 280M parameters (8.9% of target 3B model)
- M5 Max Decode Speedup
- 2.30x to 3.13x (self-reported)
- H100 Decode Speedup
- 20.4x to 2.66x (self-reported)
- M5 Max End-to-End Latency Gain
- 1.56x to 2.62x (self-reported)
- Recommended Block Size
- 8 or 9
Speculative Architecture for Vision
Liquid AI's vision drafter follows the text DSpark architecture: it taps the target model's hidden states across intermediate layers and conditions on them to propose candidate tokens. Because image patches and text tokens share a projected representation before the tapped layers, the drafter handles hidden-state vectors without modality branching. The drafter features 4 attention-only layers trained over 10 epochs on curated vision-language Supervised Fine-Tuning (SFT) data.
Benchmarks Across Edge and Server Hardware
Evaluated using the MMSpec benchmark across chart VQA, text VQA, image captioning, and multi-turn conversations:
- Apple M5 Max (MLX-VLM): 2.30x to 3.13x decode speedup, delivering 1.56x to 2.62x end-to-end latency reduction with a recommended block size of 8.
- Apple M3 Ultra (llama.cpp): 1.57x to 2.14x decode gain, reaching 1.30x to 1.77x end-to-end improvement.
- NVIDIA H100 (SGLang): 20.4x to 2.66x faster decoding, achieving 1.64x to 2.27x end-to-end acceleration with
--speculative-draft-attention-backend flashinfer.
Deployment Setup
The draft models ship with day-one support in SGLang (PR #40651), llama.cpp (PR #29339), and MLX-VLM (PR #2280). Because speculative verification is mathematically exact, greedy outputs match the standalone 3B target model bit-for-bit.
Try it in 2 minutes
llama-server -m models/LFM2.5-VL-3B-F16.gguf \
--mmproj models/mmproj-LFM2.5-VL-3B-F16.gguf \
--spec-type draft-dspark --spec-draft-n-max 8 --spec-draft-n-min 0bash
✓ When to use
- Running local OCR, chart extraction, or multi-turn vision chat on Apple Silicon via MLX or llama.cpp.
- High-throughput vision-language serving deployments on NVIDIA H100 GPUs using SGLang.
✕ When NOT to use
- Workloads where image encoding and prefill time dominate 90%+ of total latency, as speculative decoding only accelerates token generation.
- Environments with severe VRAM constraints unable to accommodate the extra 280M parameter footprint.
What to do today
- Pull the GGUF or Safetensors weights from Hugging Face for LiquidAI/LFM2.5-VL-3B-DSpark.
- Update llama.cpp or MLX-VLM builds to include native DSpark speculation support.
- Configure the speculative block size to 8 or 9 in your local inference runtime flags.
Sources