
Open platform for training, serving, and evaluating LLM chatbots
FastChat is an open-source framework by LMSYS Org for training, deploying, and benchmarking large language model chatbots. It powers Chatbot Arena with 10M+ chat requests across 70+ LLMs and provides OpenAI-compatible APIs, distributed multi-model serving, and evaluation tools like MT-Bench.
Controller/worker architecture for horizontal scaling and mixed-model deployments
Drop-in replacement for OpenAI APIs, compatible with the openai-python library
Crowdsourced evaluation platform where users compare anonymous models and vote on quality
Multi-turn benchmark with LLM-as-a-judge for automated quality assessment
Training and fine-tuning pipeline for conversational models built on LLaMA
Deploy on single GPU, multi-GPU, CPU-only, Apple Silicon, Intel XPU, or Huawei NPU
Supports ExLlama V2, GPTQ 4-bit, and AWQ 4-bit inference for consumer hardware
Organizations run an OpenAI-compatible API backed by open-source models on their own infrastructure
ML teams compare multiple LLMs using MT-Bench and human preference voting
Researchers fine-tune conversational models on custom datasets for domain-specific chatbots
Companies deploy a web UI chatbot demo without relying on external SaaS providers
Built-in web interface for testing and demoing chatbot interactions
Compatible with Llama 2, Alpaca, ChatGLM, Falcon, and dozens of other open-source LLMs
University labs study LLM behavior, alignment, and evaluation methodologies

Run AI with an API