Oleksandr Kuzmenko, EditorPublished 5 min readAI-assisted · editor-reviewed
TL;DR
- Adds speculative decoding to the 3B parameter LFM2.5-VL with an 8.9% memory overhead (280M parameters).
- Provides 1.56x to 2.62x end-to-end latency reduction on Apple Silicon and up to 2.27x on H100 GPUs.
- Guarantees exact output equivalence without quality degradation during greedy sampling.
- Ships with ready-to-run configurations for llama.cpp, MLX-VLM, and SGLang.
Key facts
- Drafter Parameter Count
- 280M parameters (8.9% of target 3B model)
- M5 Max Decode Speedup
- 2.30x to 3.13x (self-reported)
- H100 Decode Speedup
- 20.4x to 2.66x (self-reported)
- M5 Max End-to-End Latency Gain
- 1.56x to 2.62x (self-reported)
- Recommended Block Size
- 8 or 9
Source: Key facts — Recommended Block Size
Source: Key facts. Recommended Block Size: 8 or 9. Each figure is copied from the story.
- Recommended Block Size8 or 9
Speculative Architecture for Vision
Liquid AI's vision drafter follows the text DSpark architecture: it taps the target model's hidden states across intermediate layers and conditions on them to propose candidate tokens. Because image patches and text tokens share a projected representation before the tapped layers, the drafter handles hidden-state vectors without modality branching. The drafter features 4 attention-only layers trained over 10 epochs on curated vision-language Supervised Fine-Tuning (SFT) data.
Benchmarks Across Edge and Server Hardware
Evaluated using the MMSpec benchmark across chart VQA, text VQA, image captioning, and multi-turn conversations:
- Apple M5 Max (MLX-VLM): 2.30x to 3.13x decode speedup, delivering 1.56x to 2.62x end-to-end latency reduction with a recommended block size of 8.
- Apple M3 Ultra (llama.cpp): 1.57x to 2.14x decode gain, reaching 1.30x to 1.77x end-to-end improvement.
- NVIDIA H100 (SGLang): 20.4x to 2.66x faster decoding, achieving 1.64x to 2.27x end-to-end acceleration with
--speculative-draft-attention-backend flashinfer.
Deployment Setup
The draft models ship with day-one support in SGLang (PR #40651), llama.cpp (PR #29339), and MLX-VLM (PR #2280). Because speculative verification is mathematically exact, greedy outputs match the standalone 3B target model bit-for-bit.
Try it in 2 minutes
llama-server -m models/LFM2.5-VL-3B-F16.gguf \
--mmproj models/mmproj-LFM2.5-VL-3B-F16.gguf \
--spec-type draft-dspark --spec-draft-n-max 8 --spec-draft-n-min 0When to use
- Running local OCR, chart extraction, or multi-turn vision chat on Apple Silicon via MLX or llama.cpp.
- High-throughput vision-language serving deployments on NVIDIA H100 GPUs using SGLang.
When NOT to use
- Workloads where image encoding and prefill time dominate 90%+ of total latency, as speculative decoding only accelerates token generation.
- Environments with severe VRAM constraints unable to accommodate the extra 280M parameter footprint.
What to do today
- Pull the GGUF or Safetensors weights from Hugging Face for LiquidAI/LFM2.5-VL-3B-DSpark.
- Update llama.cpp or MLX-VLM builds to include native DSpark speculation support.
- Configure the speculative block size to 8 or 9 in your local inference runtime flags.
Step names are taken from What to do today.
Step names are taken from What to do today. Pull the GGUF or Safetensors weights from Hugging Face for LiquidAI/LFM2.5-VL-3B-DSpark.. Update llama.cpp or MLX-VLM builds to include native DSpark speculation support.. Configure the speculative block size to 8 or 9 in your local inference runtime flags.
- Pull the GGUF or Safetensors weights from Hugging Face for LiquidAI/LFM2.5-VL-3B-DSpark.
- Update llama.cpp or MLX-VLM builds to include native DSpark speculation support.
- Configure the speculative block size to 8 or 9 in your local inference runtime flags.