Popular repositories Loading

1. KVQuant KVQuant Public

   Forked from SqueezeAILab/KVQuant

   [NeurIPS 2024] KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization

   Python

2. BiLLM BiLLM Public

   Forked from Aaronhuang-778/BiLLM

   [ICML 2024] BiLLM: Pushing the Limit of Post-Training Quantization for LLMs

   Python

3. TensorRT-LLM TensorRT-LLM Public

   Forked from NVIDIA/TensorRT-LLM

   TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and support state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorR…

   C++

4. LLMEasyQuant LLMEasyQuant Public

   Forked from NoakLiu/LLMEasyQuant

   A Serving System for Distributed and Parallel LLM Quantization [Efficient ML System]

   Python

5. FastCache-xDiT FastCache-xDiT Public

   Forked from NoakLiu/FastCache-xDiT

   FastCache: Fast Caching for Diffusion Transformer Through Learnable Linear Approximation [Efficient ML Model]

   Python

6. PiKV PiKV Public

   Forked from NoakLiu/PiKV

   PiKV: KV Cache Management System for Mixture of Experts [Efficient ML System]

   Python