What it does
PrivateGPT is an open-source API layer for building AI applications on local models. It does not run models itself. It connects to any OpenAI-compatible inference server, such as Ollama, llama.cpp or vLLM, through an environment variable. The API follows the Claude API as its reference. It covers messages with streaming, async processing and token counting, file and document ingestion, retrieval with citations, embeddings, built-in web search, web fetch and code execution tools, custom tools, MCP connectors, and structured access to databases and CSV files. A basic version of skills is included. A workbench UI at /ui is meant for testing and demos; the project says the API is the real product and the UI is a demonstrator. Install steps are given for macOS with Homebrew, Linux and Windows with uv, and a Docker route is also mentioned. The README lists guides for using it as the backend for Claude Code, Claude Desktop, Claude for Microsoft 365, OpenCode and n8n. The project began in 2023 as a script for chatting with your documents offline. It is maintained by the team at Zylon, which sells a separate enterprise platform built on top of it.
Capabilities
- Messages API with streaming, async processing and token counting
- File ingestion and retrieval with citations
- Built-in web search, web fetch and code execution tools
- Custom tools and MCP connectors
- Database and CSV querying
- Workbench UI for testing and demos
When to use it
- You want to build AI applications on local models without depending on cloud APIs
- You already run Ollama, llama.cpp or vLLM and need an application layer on top
- You need retrieval over your own documents with citations on your own hardware
- You want tools such as Claude Code or n8n to use private models
