Alternatives to vllm-project/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

Source: vllm-project/vllm · SPDX licence: Apache-2.0

Comparable repositories

• nousresearch/vllm · A high-throughput and memory-efficient inference and serving engine for LLMs · Apache-2.0
• bentoml/bentoml · The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more! · Apache-2.0
• ollama/ollama · Get up and running with Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models. · MIT
• labring/fastgpt · FastGPT is a knowledge-based platform built on the LLMs, offers a comprehensive suite of out-of-the-box capabilities such as data processing, RAG retrieval, and visual AI workflow orchestration, letting you easily develop and deploy complex question-answering systems without the need for extensive setup or configuration. · Apache-2.0
• open-multi-agent/open-multi-agent · Self-hosted TypeScript agent runtime with durable approvals and verifiable run records. Own it, approve it, audit it. · MIT
• openinterpreter/openinterpreter · A coding agent for open models like Kimi K3 and GLM 5.3 · Apache-2.0
• huggingface/transformers · 🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training. · Apache-2.0
• ggml-org/whisper.cpp · Port of OpenAI's Whisper model in C/C++ · MIT
• unslothai/unsloth · Local UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4.1, MiniMax-H3, Gemma 4, FLUX and more. · Apache-2.0
• lmcache/lmcache · LMCache: Supercharge Your LLM with the Fastest KV Cache Layer · Apache-2.0

Comparison from the dated registry snapshot. Review each project and licence independently.