What it does
Qwen3-Omni is a natively multimodal model that handles text, images, audio and video in one system and can answer in text or in natural speech with low latency. It supports 119 text languages, 19 speech input languages and 10 speech output languages. The Apache 2.0 licensed repository describes three 30B-A3B variants: an Instruct model with both the thinking and speaking components, a Thinking model that reasons step by step and returns text, and a Captioner model fine-tuned to write detailed audio descriptions. Cookbooks walk through speech recognition, speech translation, music and sound analysis, OCR, image and video question answering, and audio-driven function calls. You can download the weights and run them with Hugging Face Transformers or vLLM, use the provided Docker image and a local web demo, or call the model through the DashScope API. The team recommends vLLM or the API for heavy or low-latency use, because Mixture-of-Experts inference in Transformers can be slow.
Capabilities
- One model for text, image, audio and video input
- Streaming text and speech output with low latency
- 119 text languages, 19 speech input languages and 10 speech output languages
- Instruct, Thinking and audio Captioner variants
- Runs with Hugging Face Transformers, vLLM or a provided Docker image
- Hosted access through the DashScope API
When to use it
- Building voice or video assistants that need to listen and reply in speech
- Transcribing, translating or captioning audio with an open model
- Self-hosting an open-weight multimodal model instead of relying on a closed API
