← Back to Discover

whisper

Open-source speech recognition model from OpenAI that transcribes multilingual audio, translates speech into English and identifies spoken languages, available in six sizes through a pip package.

TranscriptionOpen models
Visit website ↗

What it does

Whisper is a general-purpose speech recognition model released as open-source code and model weights. A single model handles multilingual speech recognition, speech translation into English and spoken language identification. It is a Transformer sequence-to-sequence model in which special tokens select the task, so one model can stand in for several stages of a traditional speech pipeline. The repository offers six sizes: tiny (39M parameters, about 1 GB of VRAM), base, small, medium, large (1550M parameters, about 10 GB) and turbo, an optimized version of large-v3 tuned for speed. The smaller sizes also come in English-only versions. You install it with pip and also need ffmpeg, then transcribe from the command line or from Python. The turbo model is not trained for translation, so turning non-English speech into English requires one of the multilingual models.

Capabilities

  • Multilingual speech recognition and translation into English
  • Spoken language identification
  • Six model sizes trading speed against accuracy and memory
  • Command-line tool and Python interface
  • MIT-licensed code and model weights

When to use it

  • Transcribing recorded audio on your own hardware
  • Building voice interfaces into applications
  • Translating non-English speech into English text