vLLM

Pinned Loading

vllm vllm Public

A high-throughput and memory-efficient inference and serving engine for LLMs

Python 64.3k 11.7k
llm-compressor llm-compressor Public

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python 2.3k 296
recipes recipes Public

Common recipes to run vLLM

Jupyter Notebook 248 94

Repositories

vllm Public
A high-throughput and memory-efficient inference and serving engine for LLMs

vllm-project/vllm’s past year of commit activity

Python 64,336 Apache-2.0 11,667 1,905 (34 issues need help) 1,259 Updated Dec 1, 2025
ci-infra Public
This repo hosts code for vLLM CI & Performance Benchmark infrastructure.

vllm-project/ci-infra’s past year of commit activity

HCL 27 Apache-2.0 46 0 24 Updated Dec 1, 2025
vllm-ascend Public
Community maintained hardware plugin for vLLM on Ascend

vllm-project/vllm-ascend’s past year of commit activity

Python 1,399 Apache-2.0 618 772 (8 issues need help) 270 Updated Dec 1, 2025
vllm-project.github.io Public

vllm-project/vllm-project.github.io’s past year of commit activity

JavaScript 24 45 0 2 Updated Dec 1, 2025
vllm-omni Public
A high-throughput and memory efficient inference and serving engine for Omni-modality models

vllm-project/vllm-omni’s past year of commit activity

Python 81 Apache-2.0 21 19 (1 issue needs help) 11 Updated Dec 1, 2025
vllm-gaudi Public
Community maintained hardware plugin for vLLM on Intel Gaudi

vllm-project/vllm-gaudi’s past year of commit activity

Python 18 Apache-2.0 73 1 67 Updated Dec 1, 2025
vllm-xpu-kernels Public
The vLLM XPU kernels for Intel GPU

vllm-project/vllm-xpu-kernels’s past year of commit activity

C++ 11 Apache-2.0 14 0 9 Updated Dec 1, 2025
tpu-inference Public
TPU inference for vLLM, with unified JAX and PyTorch support.

vllm-project/tpu-inference’s past year of commit activity

Python 171 Apache-2.0 45 16 (1 issue needs help) 62 Updated Dec 1, 2025
semantic-router Public
Intelligent Router for Mixture-of-Models

vllm-project/semantic-router’s past year of commit activity

Rust 2,345 Apache-2.0 301 109 (25 issues need help) 42 Updated Dec 1, 2025
guidellm Public
Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs

vllm-project/guidellm’s past year of commit activity

Python 726 Apache-2.0 102 42 (3 issues need help) 19 Updated Dec 1, 2025

View all repositories

Sponsors

View all

Top languages

Python C++ Rust Cuda Jupyter Notebook

Most used topics

inference llm model-serving transformer pytorch