← Back to Discover

Harbor

Command-line tool and companion app that starts a pre-connected local LLM setup with Docker: pick backends like Ollama, llama.cpp or vLLM, interfaces like Open WebUI, and extras such as web search, voice and images.

Local modelsInference & hostingCommand-line tools
Visit website ↗

What it does

Harbor is a CLI with a companion app for running local LLM stacks. A single command such as harbor up starts a default pairing of Open WebUI and llama.cpp, and further services are added by name. Harbor handles the Docker Compose orchestration, configuration and connections between services, so choosing a backend such as Ollama, llama.cpp or vLLM leaves the interface already wired to it. The catalogue spans interfaces (Open WebUI, LibreChat, SillyTavern and others), inference backends (vLLM, SGLang, Text Generation Inference, mistral.rs, MLX and more) and satellite services such as SearXNG for web search, Speaches for voice, ComfyUI for image generation, Dify, n8n and Langflow for workflows, and MCP tooling. On macOS, Docker Model Runner, MLX and oMLX run inference on the host with Metal acceleration. Harbor Boost lets you script multi-step workflows, and harbor launch wires Harbor backends into coding tools such as Codex and OpenCode. Other features include config profiles, a local command history, a QR code for opening services from a phone, a built-in tunnel for exposing services to the internet (with a warning about the risk), and an eject command that exports a standalone Docker Compose file. The README says Harbor is meant as a helper for local LLM development environments rather than a deployment solution.

Capabilities

  • One command, harbor up, starts a pre-wired stack
  • Many inference backends: Ollama, llama.cpp, vLLM, SGLang, MLX and more
  • Interfaces such as Open WebUI and LibreChat connected to the backends
  • Add-on services for web search, voice chat, image generation, RAG and workflows
  • Config profiles, and eject to a standalone Docker Compose file
  • Launch coding tools against local Harbor backends

When to use it

  • Setting up a local LLM environment without hand-writing Docker Compose files
  • Trying several inference engines or interfaces side by side
  • Experimenting with local web search, voice chat or image generation alongside a model
  • Pointing coding agents at a model running on your own machine