Skip to content

@vllm-project vLLM

Change the repository type filter

All

    Repositories list

    24 repositories

    • vllm

      Public
      A high-throughput and memory-efficient inference and serving engine for LLMs
      amdcudainferencepytorchtransformeropenaimoellamagptmodel-serving
      Python
      •
      Apache License 2.0
      •11k•60k•1.8k•1.2k•Updated Oct 13, 2025Oct 13, 2025

    • llm-compressor

      Public
      Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM
      sparsitycompressionquantization
      Python
      •
      Apache License 2.0
      •252•2.1k•56•39•Updated Oct 13, 2025Oct 13, 2025

    • vllm-gaudi

      Public
      Community maintained hardware plugin for vLLM on Intel Gaudi
      Python
      •
      Apache License 2.0
      •51•12•1•62•Updated Oct 13, 2025Oct 13, 2025

    • vllm-spyre

      Public
      Community maintained hardware plugin for vLLM on Spyre
      Python
      •
      Apache License 2.0
      •26•35•5•17•Updated Oct 13, 2025Oct 13, 2025

    • semantic-router

      Public
      Intelligent Mixture-of-Models Router for Efficient LLM Inference
      pythonkubernetesrustgolangfine-tuningenvoyproxypii-detectionmixture-of-modelshuggingface-transformersbert-classification
      Go
      •
      Apache License 2.0
      •201•1.7k•84•17•Updated Oct 13, 2025Oct 13, 2025

    • vllm-xpu-kernels

      Public
      The vLLM XPU kernels for Intel GPU
      C++
      •
      Apache License 2.0
      •14•9•0•5•Updated Oct 13, 2025Oct 13, 2025

    • vllm-ascend

      Public
      Community maintained hardware plugin for vLLM on Ascend
      inferencetransformermodel-servingmlopsascendllmllmopsllm-servingvllm
      Python
      •
      Apache License 2.0
      •480•1.2k•559•179•Updated Oct 13, 2025Oct 13, 2025

    • aibrix

      Public
      Cost-efficient and pluggable Infrastructure components for GenAI inference
      Go
      •
      Apache License 2.0
      •466•4.3k•230•24•Updated Oct 13, 2025Oct 13, 2025

    • guidellm

      Public
      Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs
      Python
      •
      Apache License 2.0
      •88•620•84•28•Updated Oct 13, 2025Oct 13, 2025

    • speculators

      Public
      A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM
      Python
      •
      Apache License 2.0
      •10•58•4•17•Updated Oct 10, 2025Oct 10, 2025

    • ci-infra

      Public
      This repo hosts code for vLLM CI & Performance Benchmark infrastructure.
      HCL
      •41•22•0•17•Updated Oct 10, 2025Oct 10, 2025

    • vllm-project.github.io

      Public
      HTML
      •29•20•0•0•Updated Oct 10, 2025Oct 10, 2025

    • production-stack

      Public
      vLLM’s reference system for K8S-native cluster-wide deployment with community-driven performance optimization
      Python
      •
      Apache License 2.0
      •304•1.8k•84•55•Updated Oct 9, 2025Oct 9, 2025

    • flash-attention

      Public
      Fast and memory-efficient exact attention
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •2k•96•0•15•Updated Oct 7, 2025Oct 7, 2025

    • recipes

      Public
      Common recipes to run vLLM
      Jupyter Notebook
      •
      Apache License 2.0
      •55•161•4•5•Updated Oct 6, 2025Oct 6, 2025

    • vllm-neuron

      Public
      Community maintained hardware plugin for vLLM on AWS Neuron
      Python
      •
      Apache License 2.0
      •0•10•0•0•Updated Oct 1, 2025Oct 1, 2025

    • FlashMLA

      Public
      C++
      •
      MIT License
      •881•6•0•3•Updated Sep 29, 2025Sep 29, 2025

    • DeepGEMM

      Public
      DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling
      Cuda
      •
      MIT License
      •711•0•0•0•Updated Sep 29, 2025Sep 29, 2025

    • vllm-openvino

      Public
      Python
      •
      Apache License 2.0
      •7•23•2•0•Updated Aug 18, 2025Aug 18, 2025

    • rfcs

      Public
      0•1•0•0•Updated Jun 3, 2025Jun 3, 2025

    • vllm-project.github.io-static

      Public archive
      HTML
      •
      MIT License
      •7•8•0•1•Updated Feb 7, 2025Feb 7, 2025

    • media-kit

      Public
      vLLM Logo Assets
      2•6•0•0•Updated Dec 12, 2024Dec 12, 2024

    • vllm-nccl

      Public archive
      Manages vllm-nccl dependency
      Python
      •
      Apache License 2.0
      •3•17•2•0•Updated Jun 3, 2024Jun 3, 2024

    • dashboard

      Public
      vLLM performance dashboard
      Python
      •
      Apache License 2.0
      •7•36•0•0•Updated Apr 26, 2024Apr 26, 2024