FastChat
Open platform for training, serving, and evaluating LLM chatbots
Founded
2023
Starting Price
\u00240
Category
AI Chatbots & Agents
Last updated March 6, 2026
About FastChat
FastChat is an open-source framework by LMSYS Org for training, deploying, and benchmarking large language model chatbots. It powers Chatbot Arena with 10M+ chat requests across 70+ LLMs and provides OpenAI-compatible APIs, distributed multi-model serving, and evaluation tools like MT-Bench.
Pros & Cons
Pros
- Fully open-source with Apache 2.0 license for complete transparency
- OpenAI-compatible API enables seamless migration from proprietary to self-hosted models
- Supports 70+ LLMs with broad hardware flexibility including GPU, CPU, and Apple Silicon
- Battle-tested at scale powering Chatbot Arena with 10M+ requests
- Comprehensive evaluation tools including MT-Bench and LLM-as-a-judge
Key Features
Distributed Multi-Model Serving
Controller/worker architecture for horizontal scaling and mixed-model deployments
OpenAI-Compatible API
Drop-in replacement for OpenAI APIs, compatible with the openai-python library
Chatbot Arena
Crowdsourced evaluation platform where users compare anonymous models and vote on quality
MT-Bench Evaluation
Multi-turn benchmark with LLM-as-a-judge for automated quality assessment
Vicuna Model Training
Training and fine-tuning pipeline for conversational models built on LLaMA
Broad Hardware Support
Deploy on single GPU, multi-GPU, CPU-only, Apple Silicon, Intel XPU, or Huawei NPU
Quantized Inference
Supports ExLlama V2, GPTQ 4-bit, and AWQ 4-bit inference for consumer hardware
Pricing
Open Source
\u00240
- Full platform
- Training + serving + evaluation
- Web UI
- OpenAI-compatible API
- All model support
Best For
Self-Hosted LLM API Server
Organizations run an OpenAI-compatible API backed by open-source models on their own infrastructure
Model Evaluation & Benchmarking
ML teams compare multiple LLMs using MT-Bench and human preference voting
LLM Fine-Tuning
Researchers fine-tune conversational models on custom datasets for domain-specific chatbots
Internal Chatbot Prototyping
Companies deploy a web UI chatbot demo without relying on external SaaS providers
