vLLM
High-throughput and memory-efficient inference engine for LLMs. Features PagedAttention and continuous batching for production serving.
Category: Inference · Pricing: Open Source
High-throughput and memory-efficient inference engine for LLMs. Features PagedAttention and continuous batching for production serving.
Category: Inference · Pricing: Open Source