Harbor
Run a complete local LLM stack with one command
Founded
2024
Starting Price
\u00240
Category
Developer Tools
Last updated March 8, 2026
About Harbor
Harbor is a containerized CLI toolkit that brings a complete pre-wired LLM stack with hundreds of services to your local machine. It handles Docker Compose orchestration, configuration, and cross-service connectivity so you can run LLM backends like Ollama, llama.cpp, or vLLM alongside frontends like Open WebUI and supporting services like SearXNG, ComfyUI, and LangFuse — all with minimal configuration.
Pros & Cons
Pros
- Single command spins up an entire local AI development environment with dozens of pre-wired services
- Supports all major LLM inference engines and frontends with automatic cross-service connectivity
- Shared cache system prevents redundant downloads of large model files across services
- Completely free and open source under MIT license with active development
- Harbor Boost modules add middleware capabilities like caching, routing, and request modification
Key Features
One-Command Setup
Spin up a complete local LLM stack with a single harbor up command — backends, frontends, and services all pre-wired together
100+ Supported Services
Supports major inference engines (Ollama, llama.cpp, vLLM, TGI, LiteLLM, TabbyAPI, Aphrodite, SGLang) plus dozens of frontends and utilities
Pre-Configured Integrations
Services are automatically connected — running SearXNG enables Web RAG in Open WebUI, Speaches provides OpenAI-compatible STT and TTS out of the box
Harbor App UI
A companion desktop app providing a clean interface for managing services, configurations, and workflows
Harbor Boost
Modular enhancement system that adds capabilities like caching, routing, and middleware to your LLM stack
Harbor Bench
Built-in benchmarking tool for evaluating and comparing LLM performance across different backends and models
Shared Cache System
Host cache is shared and reused across services — Hugging Face models, Ollama weights, and other artifacts are downloaded once
Pricing
Free
\u00240
- All services included
- Unlimited local usage
- Community support
- MIT license
- Harbor App UI
Best For
Local LLM Development
Set up a complete local AI development environment with inference engines, frontends, and supporting services for model experimentation
AI Service Evaluation
Quickly spin up and compare different LLM backends, frontends, and tooling to find the best combination for your workflow
Offline AI Workflows
Run AI models and services entirely locally for privacy-sensitive work without sending data to external APIs
AI Tool Prototyping
Rapidly prototype applications using local LLMs with pre-connected RAG, image generation, voice chat, and code assistance services
