Thinking Machines Releases Inkling Small 12B Active Open-Weight Multimodal Model
Thinking Machines Lab released Inkling Small, an open-weight multimodal mixture-of-experts model with 12B active parameters out of 276B total. It achieves a 40.2 Artificial Analysis intelligence score and offers self-hosting options alongside cheap API pricing of $0.50 input and $1.20 output per million tokens.
Impact: Medium
Why it matters
Developers can deploy a performant multimodal MoE model locally via vLLM or Ollama to cut API token costs to zero for code generation.
TL;DR
- 01Inkling Small provides 12B active parameters out of 276B total in an open-weight MoE format.
- 02Self-hosting via vLLM or Ollama eliminates per-token operational costs completely.
- 03Hosted API pricing is set at $0.50 input and $1.20 output per 1M tokens.
Key facts
- Active Parameters
- 12B
- Total Parameters
- 276B
- Hosted Input Price
- $0.50 / 1M tokens
- Hosted Output Price
- $1.20 / 1M tokens
- Generation Speed
- 124 tokens/sec
- GPQA Score
- 89.5%
Open-Weight MoE Architecture
Inkling Small by Thinking Machines Lab utilizes a Mixture-of-Experts architecture with 12 billion active parameters out of 276 billion total parameters. The open-weight release allows self-hosting on private hardware without incurring per-token API charges.
Performance Benchmarks and Inference Speed
According to independent benchmarks by Artificial Analysis, the model achieves an intelligence index score of 40.2. Specific scores include 89.5% on GPQA, 31.6% on Humanity's Last Exam, 63% on Long Context Reasoning, 48.7% on SciCode, 15.5% on τ-Bench Banking, and 55.1% on Terminal-Bench. Hosted inference streams at approximately 124 tokens per second with a median initial token delay of 1.44 seconds.
Pricing Structure
For developers preferring hosted endpoints, list pricing is set at $0.50 per million input tokens and $1.20 per million output tokens, resulting in a blended operational cost of approximately $0.68 per million tokens under standard input-to-output ratios.
Try it in 2 minutes
ollama run inkling-smallbash
✓ When to use
- Self-hosting code generation agents on local GPU clusters without per-token fees.
- Replacing expensive proprietary API models for high-throughput multimodal tasks.
- Running Terminal-Bench style terminal automation agents requiring fast execution.
✕ When NOT to use
- Deploying on hardware with insufficient VRAM to fit a 276B total parameter MoE model.
- Complex banking or specialized financial agent workflows requiring high precision (τ-Bench Banking is 15.5%).
What to do today
- Test local deployment using Ollama or vLLM on your GPU infrastructure.
- Benchmark execution speeds against current code generation models on Terminal-Bench tasks.
Sources