Alibaba Launches Qwen3.8-LiveTranslate Realtime WebSocket Interpretation Model
Alibaba released Qwen3.8-LiveTranslate, a real-time multimodal interpretation model that cuts average lag to 2.3 seconds across 60 languages. Developers can integrate it via a streaming WebSocket endpoint with native speaker diarization.
Impact: Medium
Why it matters
You can now stream bidirectional voice and video translation directly into agent workflows with predictable token costs and under 2.5-second response times.
TL;DR
- 01Cuts live speech-to-speech translation lag to 2.3 seconds using an interleaved streaming architecture.
- 02Processes 60 input languages with 29 spoken voice output languages over WebSocket.
- 03Consumes 7 input and 12.5 output tokens per second, costing around $1.54 per hour of continuous conversation.
Key facts
- Average Lagging (LAAL)
- 2.3 seconds (vendor-reported, down from 2.8s)
- Language Coverage
- 60 understanding, 29 speech output
- Context Window
- 53,248 tokens (49,152 input / 4,096 output)
- Token Consumption
- 7 tokens/sec audio in, 12.5 tokens/sec audio out
- Audio Specifications
- 16 kHz PCM input, 24 kHz PCM output
Architecture and Latency Metrics
Qwen3.8-LiveTranslate runs on an Interleave architecture that decodes text and audio simultaneously while ingesting continuous audio streams. Length-Adaptive Average Lagging (LAAL) is reduced to 2.3 seconds, down from 2.8 seconds in prior iterations. The architecture includes long-context disambiguation across a 53,248-token context window (49,152 tokens reserved for input, 4,096 for output).
Audio Specifications and Pricing
The WebSocket Realtime API expects 16 kHz PCM audio input and generates 24 kHz PCM output streams. Ingestion consumes 7 audio tokens per second, while audio generation consumes 12.5 tokens per second. An hour of bidirectional voice interpretation costs approximately $1.54 on Singapore nodes before factoring in text and image tokens.
Multimodal and Diarization Capabilities
Beyond speech, the model ingests up to 2 image frames per second to interpret visual cues like lip motion and presentation slides. It performs real-time speaker diarization and provides an always voice-cloning mode that re-clones voices on every speaker transition. Custom glossaries support up to 1,000 hotwords.
Try it in 2 minutes
{
"model": "qwen3.8-livetranslate-flash-realtime",
"turn_detection": {
"type": "speaker_detection"
},
"session": {
"output_modalities": ["text", "audio"]
}
}json
✓ When to use
- Live international team calls or multilingual voice bot interfaces requiring low-latency audio feedback.
- Multimodal agent streams where visual gestures or on-screen slides assist conversational understanding.
✕ When NOT to use
- Do not use for batch document translation, structured JSON generation, or function calling workflows.
- Avoid when you require fully self-hosted, offline inference, as the realtime model is API-only.
What to do today
- Connect to the WebSocket endpoint using model ID qwen3.8-livetranslate-flash-realtime.
- Set audio format to 16 kHz PCM input and handle 24 kHz PCM binary chunks.
- Send session.finish before terminating connections to prevent loss of the final audio segment.
Sources