Apertus 1.5 Adds Native Multimodal Tokens, Thinking Mode, and 262K Context
Swiss AI released Apertus 1.5 in 8B and 70B parameter sizes under the Apache 2.0 license, introducing native image and audio processing alongside a chain-of-thought thinking mode. The release expands the context window to 262,144 tokens while publishing full training data receipts, intermediate checkpoints, and standardized evaluation cards.
Why it matters
You can now self-host an open-source, fully auditable reasoning and multimodal model with a 262k context window without restrictive commercial licensing.
TL;DR
- 01Apertus 1.5 8B and 70B weights, datasets, and intermediate checkpoints are released under Apache 2.0.
- 02Early-fusion tokenization maps 512x512 images to 1,024 tokens and audio to 40 tokens per second.
- 03The context window expands fourfold to 262,144 tokens with an optional reasoning thinking mode.
Key facts
- Context Window
- 262,144 tokens
- Model Variants
- 8B and 70B parameters
- Total Pretraining Volume
- 19T tokens (8B) / 17T tokens (70B)
- Image Token Density
- 1,024 tokens per 512x512 input
- Audio Token Rate
- 40 tokens per second
- Software License
- Apache 2.0
Early-Fusion Architecture
Apertus 1.5 avoids external vision cross-attention adapters by mapping multimodal streams directly into discrete tokens within the primary vocabulary. Images processed via the Emu3.5 tokenizer convert a standard 512x512 matrix into 1,024 discrete tokens, while spoken audio enters the context sequence via WavTokenizer at a density of 40 tokens per second. The model processes multimodal inputs natively but restricts text generation outputs to standard tokens.
Continued Pretraining and Context Scaling
Training scaled the original foundation checkpoints with an additional 4 trillion tokens for the 8B variant and 2 trillion tokens for the 70B model, reaching cumulative datasets of 19T and 17T tokens. The training curriculum progressed through four explicit phases: 1. Unpaired raw text and standalone image/audio sequences 2. Paired image-caption and transcript data alignments 3. High-quality code and mathematical cooldown mixtures 4. Progressive context expansion stepping from 8,192 up to 262,144 tokens
Verifiable Post-Training and Tool Use
Post-training integrates 3.7 million supervised fine-tuning examples with verified tool execution traces, stripping hallucinated API calls. Reinforcement Learning with Verifiable Rewards (RLVR) was executed across 350,000 tasks covering code, mathematics, and vision reasoning. Users can toggle the integrated thinking mode to allocate dynamic reasoning tokens prior to final generation.
Try it in 2 minutes
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "swiss-ai/Apertus-1.5-8B-Instruct"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto", device_map="auto")
messages = [{"role": "user", "content": "Analyze this architecture diagram."}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))python
✓ When to use
- Deploy when your pipeline requires fully auditable open weights and permissive Apache 2.0 licensing.
- Use for complex long-context document analysis requiring combined image, audio, and reasoning steps.
✕ When NOT to use
- Do not use when direct audio synthesis or image generation outputs are required.
- Avoid running the unquantized 70B model on systems with less than 160 GB VRAM.
What to do today
- Inspect the Apertus 1.5 model collection on Hugging Face to evaluate Apache 2.0 weights.
- Test local deployment of the 8B checkpoint for document parsing and reasoning benchmarks.
- Verify tool-use capabilities using the published verified SFT trajectory datasets.