Local LLMs
Bonsai 2 27B Delivers Near-Lossless Ternary Compression for Coding Agents
Prism ML released Ternary Bonsai 2 27B, quantizing Qwen3.8 27B down to 1.76 bits per weight at a 5.9GB memory footprint. The release retains 98.2% of full-precision benchmark performance while running local agentic coding loops on consumer GPUs.
September 18, 2026 4 min read
Curated by Oleksandr Kuzmenko, AI Product EngineerUpdated September 18, 2026Sources cited on every story
AI-assisted · editor-reviewedHow we use AI
Impact: High
Why it matters
You can now run a capable 27B coding and computer-use agent completely offline on consumer hardware with 6GB of VRAM.
TL;DR
- 01Runs full 27B reasoning and multimodal computer use within 5.9GB of system memory.
- 02Retains 98.2% baseline accuracy across BigCodeBench, LiveCodeBench v6, and HumanEval+.
- 03Delivers 143 tokens per second on an RTX 5090 and 46.8 tokens per second on Apple M5 Max.
Key facts
- Model Size
- 5.9GB
- Effective Bits
- 1.76 bits/weight
- Context Window
- 262K tokens
- Benchmark Retention
- 98.2% (self-reported)
- Throughput (RTX 5090)
- 143 tokens/sec
- License
- Apache 2.0
Extreme Quantization Without Agent Drift Ternary Bonsai 2 27B, built on top of Qwen3.8 27B, reduces model size by over 9x to 5.9GB using 1.76 effective bits per weight. While previous sub-2-bit experiments failed on multi-turn coding benchmarks due to cascading logic errors, Bonsai 2 scores 83.9 on aggregate benchmarks, retaining 98.2% of its full-precision baseline. ### Hardware Throughput and Agent Integration The model achieves 143 tokens/s on desktop NVIDIA RTX 5090 cards and 46.8 tokens/s on Apple M5 Max hardware via native MLX kernels. With an energy draw of 0.714 mWh per token on an RTX 4090, it operates at 40% lower energy consumption than a standard full-precision 8B parameter model. Prism ML verified compatibility with Cline agent loops and multimodal computer-use tasks across a 262K context window under Apache 2.0.
Try it in 2 minutes
git clone https://github.com/prismml/bonsai && cd bonsai && pip install -e . && python -m bonsai.serve --model ternary-bonsai-2-27b --context-window 262144bash
✓ When to use
- Running offline agentic coding pipelines with strict privacy requirements.
- Hosting multi-turn tool-calling loops on consumer laptops or single GPU workstations.
- Budgeting long-running background reasoning where cloud API costs are prohibitive.
What to do today
- Download the model weights from the official Apache 2.0 repository.
- Benchmark local inference speed using the custom MLX kernel on Apple Silicon or CUDA on RTX GPUs.
- Hook the local endpoint into Cline or Cursor to test multi-turn agentic editing without cloud fees.
Sources