What it does
TurboPilot is a self-hosted code completion server that uses the library behind llama.cpp to run code models locally on CPU. It serves completions over an OpenAI-style API on port 18080, and the author recommends the vscode-fauxpilot plugin as the editor front end. You can also call the completions endpoint directly with a POST request. It runs as a downloaded binary or from a Docker image, with CUDA-enabled images and a Linux executable for GPU offload. Models are downloaded pre-quantized from Hugging Face or converted yourself. Supported families include StableCode 3B Instruct, WizardCoder, Starcoder, Santacoder and the first-generation Codegen models. Only one GPU device is used at a time. The repository states that the project was deprecated and archived on 30 September 2023 and is now read-only, and that other mature solutions meet the community's needs better. It was also described as a proof of concept with slow autocompletion.
Capabilities
- Runs code completion models locally on CPU using the library behind llama.cpp
- Supports StableCode 3B Instruct, WizardCoder, Starcoder, Santacoder and Codegen models in quantized form
- Optional CUDA GPU offload through Docker images or the Linux executable
- Completions endpoint modelled on the OpenAI-style API
- Works from VS Code through the vscode-fauxpilot plugin
When to use it
- You want to experiment with fully local code completion on a machine without a large GPU
- You are studying an early llama.cpp-based completion server and accept that it is no longer maintained
