Pinned Loading

1. vllm vllm Public

   Forked from vllm-project/vllm

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python

2. ray ray Public

   Forked from ray-project/ray

   Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

   Python

3. llm-d llm-d Public

   Forked from llm-d/llm-d

   llm-d is a Kubernetes-native high-performance distributed LLM inference framework

   Makefile

4. dynamo dynamo Public

   Forked from ai-dynamo/dynamo

   A Datacenter Scale Distributed Inference Serving Framework

   Rust

5. nixl nixl Public

   Forked from ai-dynamo/nixl

   NVIDIA Inference Xfer Library (NIXL)

   C++

6. LMCache LMCache Public

   Forked from LMCache/LMCache

   Supercharge Your LLM with the Fastest KV Cache Layer

   Python