Oleksandr Kuzmenko, EditorPublished 6 min readAI-assisted · editor-reviewed
TL;DR
- Phonon 31m provides fully offline, Apache 2.0 text-to-speech in a 65 MB footprint.
- Outperforms models up to 100x its size on German and Spanish intelligibility on the Open TTS Leaderboard.
- Runs on device CPUs, WebGPU, Apple Silicon, and CUDA via ptts and Transformers v5.21+.
Key facts
- Model size (31m / 40m)
- 65 MB / 75 MB
- Parameter count
- 30,788,354 parameters (audio tokenizer: 10,305,313)
- Phonon 31m license
- Apache 2.0 (commercial use permitted)
- Phonon 40m license
- CC BY-NC (seat-based commercial licensing)
- Supported languages
- English, French, German, Spanish, Portuguese
- Adoption milestone
- 200,000+ downloads in first two weeks
Source: Key facts — Model size (31m / 40m)
Source: Key facts. Model size (31m / 40m): 65 MB / 75 MB. Each figure is copied from the story.
- Model size (31m / 40m)65 MB / 75 MB
Architecture and Flow-Matching Latents
Phonon relies on flow-matching generative mechanics derived from the Pocket TTS architecture. Unlike conventional acoustic models, it generates compressed audio latents autoregressively, treating audio frames equivalently to next-token predictions in language models. The audio tokenizer acts solely as a decoder during inference, while speaker identity is injected via precomputed speaker latent tensors rather than raw waveform reference inputs.
Model Sizes and Licensing
Phonon is distributed in two primary checkpoints:
- Phonon 31m (~65 MB): Fully open under the Apache 2.0 license. Ships with full commercial rights for English voice generation.
- Phonon 40m (~75 MB): Covers 5 languages (English, French, German, Spanish, and Portuguese) licensed under CC BY-NC. Commercial licensing uses active user seat counts rather than generated speech duration.
Over 100 precomputed voices are published in the gradium/phonon-voices Hugging Face repository, and custom speaker latents can be generated using Gradium Studio.
Inference Stack: ptts and Transformers
Gradium built ptts, an inference framework targeting low time-to-first-audio across single-stream devices. It supports WebAssembly and WebGPU in browsers, Apple Core ML and Snapdragon NPUs on mobile devices, and CUDA/Metal on developer workstations. For server-side synthesis or dataset generation, Phonon integrates natively into 🤗 Transformers from version v5.21, supporting batched generation and full-graph torch.compile acceleration.
Try it in 2 minutes
from transformers import AutoProcessor, GradiumPhononTTSForConditionalGeneration
from huggingface_hub import hf_hub_download
from safetensors.torch import load_file
model_id = "gradium/phonon-31m"
processor = AutoProcessor.from_pretrained(model_id)
model = GradiumPhononTTSForConditionalGeneration.from_pretrained(model_id, device_map="auto")
voice_file = hf_hub_download("gradium/phonon-voices", "voices/LMDIYE0Oa377Qh6Z.safetensors")
voice = load_file(voice_file)["speaker_wavs"][0].T
conversation = [{"role": "user", "content": [{"type": "audio"}, {"type": "text", "text": "Local voice agent active."}]}]
inputs = processor(conversation=conversation, audio_prompt_latents=[voice], return_tensors="pt").to(model.device)
outputs = model.generate(**inputs)
audio = processor.decode(outputs.audio_latents, outputs.audio_latents_mask)
processor.save_audio(audio, "output.wav")When to use
- When building conversational local agents that must function completely offline on laptops, smartwatches, or single-board devices.
- When zero-latency speech synthesis is required inside client-side browser apps using WebGPU or WebAssembly.
When NOT to use
- Do not use Phonon 40m in commercial software products without purchasing a commercial seat license from Gradium.
- Avoid using Phonon for languages outside English, French, German, Spanish, and Portuguese until dedicated checkpoints are trained.
What to do today
- Install transformers>=5.21 to test local speech synthesis with the Phonon 31m checkpoint.
- Download precomputed speaker latents from gradium/phonon-voices to evaluate voice styles without retraining.
- Benchmark ptts WebGPU inference inside local browser extensions or Electron wrappers for zero-cloud voice interfaces.
Step names are taken from What to do today.
Step names are taken from What to do today. Install transformers>=5.21 to test local speech synthesis with the Phonon 31m checkpoint.. Download precomputed speaker latents from gradium/phonon-voices to evaluate voice styles without retraining.. Benchmark ptts WebGPU inference inside local browser extensions or Electron wrappers for zero-cloud voice interfaces.
- Install transformers>=5.21 to test local speech synthesis with the Phonon 31m checkpoint.
- Download precomputed speaker latents from gradium/phonon-voices to evaluate voice styles without retraining.
- Benchmark ptts WebGPU inference inside local browser extensions or Electron wrappers for zero-cloud voice interfaces.