Repositories list

• sglang

  Public
  SGLang is a fast serving framework for large language models and vision language models.

  cudainferencepytorch+ 17transformeropenaimoellamavlmkimiblackwell+ 10

  Python

  •

  Apache License 2.0

  •2.5k•16k•549•538•Updated Aug 2, 2025Aug 2, 2025

• sgl-project.github.io

  Public
  This is the documentation repository for SGLang. It is auto-generated from https://github.com/sgl-project/sglang/tree/main/docs.

  Jupyter Notebook

  •15•63•7•4•Updated Aug 1, 2025Aug 1, 2025

• ome

  Public
  OME is a Kubernetes operator for enterprise-grade management and serving of Large Language Models (LLMs)

  k8sllamaoracle-cloud+ 9model-servingmodel-as-a-servicemulti-node-kubernetesllmllm-inferencedeepseeksglang+ 2

  Go

  •

  MIT License

  •34•202•25•17•Updated Aug 1, 2025Aug 1, 2025

• genai-bench

  Public
  Genai-bench is a powerful benchmark tool designed for comprehensive token-level performance evaluation of large language model (LLM) serving systems.

  Python

  •

  MIT License

  •15•180•6•3•Updated Aug 1, 2025Aug 1, 2025

• sgl-kernel-xpu

  Public
  SGLang kernel library for Intel XPU

  Python

  •

  MIT License

  •1•6•0•0•Updated Aug 1, 2025Aug 1, 2025

• SpecForge

  Public
  Train speculative decoding models effortlessly and port them smoothly to SGLang serving.

  Python

  •

  MIT License

  •37•263•15•8•Updated Aug 1, 2025Aug 1, 2025

• sgl-kernel-npu

  Public
  SGLang kernel library for NPU

  C++

  •

  MIT License

  •6•19•1•4•Updated Jul 31, 2025Jul 31, 2025

• whl

  Public
  Kernel Library Wheel for SGLang

  cudacutlasssglang+ 1flashinfer

  HTML

  •

  MIT License

  •2•10•0•0•Updated Jul 31, 2025Jul 31, 2025

• sgl-learning-materials

  Public
  Materials for learning SGLang

  MIT License

  •38•516•0•0•Updated Jul 19, 2025Jul 19, 2025

• llama-stack-evals

  Public
  Functional tests and benchmarks for verifying Llama model providers.

  Python

  •

  MIT License

  •13•1•0•0•Updated Jul 3, 2025Jul 3, 2025

• DeepGEMM

  Public
  DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling

  Python

  •

  MIT License

  •666•17•0•0•Updated Jun 12, 2025Jun 12, 2025

• torchtitan

  Public
  A PyTorch native library for large-scale model training

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •451•0•0•0•Updated May 18, 2025May 18, 2025

• sgl-attn

  Public
  Fast and memory-efficient exact attention

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •1.9k•10•0•1•Updated May 15, 2025May 15, 2025

• sgl-test-files

  Public
  The test files for SGLang.

  MIT License

  •1•0•0•0•Updated Mar 12, 2025Mar 12, 2025

• tensorrt-demo

  Public
  TensorRT LLM Benchmark Configuration

  Python

  •4•13•0•0•Updated Jul 26, 2024Jul 26, 2024