Skip to content

@kvcache-ai kvcache.ai

Change the repository type filter

All

    Repositories list

    7 repositories

    • Mooncake

      Public
      Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.
      inferencerdmadisaggregationllmvllmsglangkvcache
      C++
      •
      Apache License 2.0
      •361•3.8k•125•47•Updated Aug 27, 2025Aug 27, 2025

    • sglang

      Public
      SGLang is a fast serving framework for large language models and vision language models.
      Python
      •
      Apache License 2.0
      •2.7k•2•0•1•Updated Aug 24, 2025Aug 24, 2025

    • sglang-npu

      Public
      SGLang is a fast serving framework for large language models and vision language models.
      Python
      •
      Apache License 2.0
      •2.7k•0•0•0•Updated Aug 12, 2025Aug 12, 2025

    • ktransformers

      Public
      A Flexible Framework for Experiencing Cutting-edge LLM Inference Optimizations
      Python
      •
      Apache License 2.0
      •1.1k•15k•603•16•Updated Aug 2, 2025Aug 2, 2025

    • DeepEP_fault_tolerance

      Public
      DeepEP: an efficient expert-parallel communication library that supports fault tolerance
      Cuda
      •
      MIT License
      •907•1•0•0•Updated Jul 31, 2025Jul 31, 2025

    • custom_flashinfer

      Public
      FlashInfer: Kernel Library for LLM Serving
      Cuda
      •
      Apache License 2.0
      •462•6•0•0•Updated Jul 24, 2025Jul 24, 2025

    • vllm

      Public
      A high-throughput and memory-efficient inference and serving engine for LLMs
      Python
      •
      Apache License 2.0
      •9.7k•14•0•0•Updated Mar 27, 2025Mar 27, 2025