What it does
whisper.cpp is a plain C/C++ implementation of OpenAI's Whisper speech recognition model for running inference without external dependencies. The high-level model code lives in whisper.h and whisper.cpp, with the rest coming from the ggml machine learning library, which keeps it easy to embed in other applications. It targets a wide range of hardware: Apple Silicon (ARM NEON, Accelerate, Metal and Core ML), x86 AVX, POWER VSX, NVIDIA CUDA, AMD ROCm, Vulkan, OpenVINO and several NPUs, as well as CPU-only runs. It supports mixed F16/F32 precision and integer quantization, and quantized models need less memory and disk space. The README shows it running fully offline on an iPhone. The quick start is to clone the repo, download a ggml-format model, build with CMake and run the whisper-cli example on a 16-bit WAV file. The repository also includes a real-time microphone streaming example, an HTTP server example, Docker images, voice activity detection and experimental word-level timestamps. The README states that it is inference only.
Capabilities
- Dependency-free C/C++ inference for Whisper models
- Apple Silicon support through Metal and Core ML
- GPU and NPU backends including CUDA, Vulkan, ROCm and OpenVINO
- Integer quantization to shrink models
- Real-time microphone transcription example
- Docker images and an HTTP server example
When to use it
- Running speech-to-text fully offline on a laptop or phone
- Embedding Whisper transcription in a native application
- Running Whisper on limited or non-NVIDIA hardware
