What it does

Retrieval-based Voice Conversion WebUI is a project for turning one voice into another. You train a conversion model on your own voice data, then use it through a browser-based interface that runs on your own machine. The repository says roughly ten minutes of clean recording is enough to start, and recommends gathering at least that much low-noise speech. To keep the original speaker's tone from bleeding through, it swaps input features for the closest matches found in the training set. Trained models can also be blended together, and the project can call pymss/MSST models to separate vocals from accompaniment. Those separation weights are downloaded separately and are needed only for that feature. Pitch is extracted with a method called RMVPE. A second window handles real-time voice changing. The page reports about 170 ms end-to-end delay, or about 90 ms with ASIO audio devices, depending heavily on hardware drivers. Setup is done by hand with Python 3.12, with separate dependency files for CPU-only, older NVIDIA and RTX 50 series cards, and models are downloaded from a Hugging Face repository. The README is mainly written in Chinese, with links to translations. The project is MIT licensed.

Capabilities

Train a conversion model from about ten minutes of voice data
Retrieval step that replaces input features with training-set matches to limit tone leakage
Real-time voice changer window with reported 170 ms end-to-end delay
Merge trained models to change the resulting voice
Optional vocal and accompaniment separation
Runs with NVIDIA GPUs, or on CPU, AMD and Intel hardware

When to use it

You want to train a voice conversion model on your own recordings
You need a live voice changer that runs on your own computer
You are comfortable installing Python dependencies and downloading model files manually