vLLM

vLLM is the most popular open-source LLM inference engine. It implements PagedAttention for efficient memory management, achieving 2-4x higher throughput than naive serving. vLLM supports continuous batching, tensor parallelism, speculative decoding, and serves an OpenAI-compatible API. It's the standard engine behind most self-hosted LLM deployments.