Alternatives to lyogavin/airllm

AirLLM 2.8T LLM inference with single 4GB GPU

Source: lyogavin/airllm · SPDX licence: Apache-2.0

Comparable repositories

• vllm-project/vllm · A high-throughput and memory-efficient inference and serving engine for LLMs · Apache-2.0
• bentoml/bentoml · The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more! · Apache-2.0
• nousresearch/nous-llama.cpp · LLM inference in C/C++ - nous research · MIT
• bytedance/infinistore · KV cache store for distributed LLM inference · Apache-2.0
• ggml-org/llama.cpp · LLM inference in C/C++ · MIT
• richardgill/llm-ui · The React library for LLMs · MIT
• nousresearch/speculators · A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM · Apache-2.0

Comparison from the dated registry snapshot. Review each project and licence independently.