Repositories list

• vllm

  Public
  vLLM: A high-throughput and memory-efficient inference and serving engine for LLMs

  inferencepytorchtransformer+ 6gptamdgpurocmmodel-servingllmllm-inference

  Python

  •

  Apache License 2.0

  •9.1k•87•28•3•Updated Aug 5, 2025Aug 5, 2025

• vllm-rocmfork

  Public
  A high-throughput and memory-efficient inference and serving engine for LLMs

  Python

  •

  Apache License 2.0

  •9.1k•0•0•0•Updated Aug 1, 2025Aug 1, 2025

• vllmtests

  Public
  This is a repository containing the tools for testing vLLM correctness and perf regression

  Python

  •

  Apache License 2.0

  •0•0•0•0•Updated Jul 30, 2025Jul 30, 2025

• LMCache

  Public
  ROCm support of Ultra-Fast and Cheaper Long-Context LLM Inference

  Python

  •

  Apache License 2.0

  •449•0•1•0•Updated Jul 15, 2025Jul 15, 2025

• roxl

  Public
  NVIDIA Inference Xfer Library (NIXL)

  C++

  •

  Apache License 2.0

  •117•0•0•0•Updated Jun 6, 2025Jun 6, 2025

• aiter

  Public
  AI Tensor Engine for ROCm

  Python

  •

  MIT License

  •83•0•0•0•Updated Jun 5, 2025Jun 5, 2025

• JamAIBase

  Public
  The collaborative spreadsheet for AI. Chain cells into powerful pipelines, experiment with prompts and models, and evaluate LLM responses in real-time. Work together seamlessly to build and iterate on AI applications.

  pythonworkflowai+ 17serverlesschatbotspreadsheetsvelteorchestrationbaasagents+ 10

  Python

  •

  Apache License 2.0

  •33•1k•1•0•Updated May 29, 2025May 29, 2025

• aiter-api-watcher

  Public
  This is a repository to monitor the fast changing ROCm/aiter repository to alert user that AITER function of interests e.g. in vLLM, in SGLang has been updated at certain commit.

  Python

  •

  Apache License 2.0

  •0•0•39•0•Updated Apr 27, 2025Apr 27, 2025

• vllmWorkshop

  Public
  vLLM Workshop Content

  Apache License 2.0

  •0•2•0•0•Updated Apr 3, 2025Apr 3, 2025

• amd-gpu-workshops

  Public
  Jupyter Notebook

  •4•0•0•0•Updated Mar 20, 2025Mar 20, 2025

• jamaibase-ts-docs

  Public
  Typescript Documentation of JamAISDK

  HTML

  •0•0•0•0•Updated Mar 14, 2025Mar 14, 2025

• git-version-tutorial

  Public
  Python

  •

  Apache License 2.0

  •1•0•0•0•Updated Feb 24, 2025Feb 24, 2025

• lmcache-vllm

  Public
  The driver for LMCache core to run in vLLM

  Python

  •

  Apache License 2.0

  •30•0•0•0•Updated Jan 24, 2025Jan 24, 2025

• lmcache-tests

  Public
  Python

  •8•0•0•0•Updated Jan 23, 2025Jan 23, 2025

• production-stack

  Public
  Python

  •

  Apache License 2.0

  •249•0•0•0•Updated Jan 22, 2025Jan 22, 2025

• kvpress

  Public
  LLM KV cache compression made easy

  Python

  •

  Apache License 2.0

  •51•0•0•0•Updated Jan 21, 2025Jan 21, 2025

• litellm

  Public
  Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq]

  Python

  •

  Other

  •3.7k•0•0•0•Updated Jan 13, 2025Jan 13, 2025

• composable_kernel

  Public
  Composable Kernel: Performance Portable Programming Model for Machine Learning Tensor Operators

  C++

  •

  Other

  •220•0•0•0•Updated Dec 20, 2024Dec 20, 2024

• Mooncake

  Public
  Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

  C++

  •

  Apache License 2.0

  •339•0•0•0•Updated Dec 16, 2024Dec 16, 2024

• torchac_rocm

  Public
  ROCm Implementation of torchac_cuda from LMCache

  Cuda

  •6•0•0•0•Updated Dec 16, 2024Dec 16, 2024

• etalon

  Public
  LLM Serving Performance Evaluation Harness

  Python

  •

  Apache License 2.0

  •11•0•0•0•Updated Dec 16, 2024Dec 16, 2024

• infinity-executable

  Public
  Infinity is a high-throughput, low-latency REST API for serving vector embeddings, supporting a wide range of text-embedding models and frameworks.

  Python

  •

  MIT License

  •159•0•0•1•Updated Dec 7, 2024Dec 7, 2024

• Liger-Kernel

  Public
  Efficient Triton Kernels for LLM Training

  Python

  •

  BSD 2-Clause "Simplified" License

  •379•0•0•0•Updated Dec 6, 2024Dec 6, 2024

• Star-Attention

  Public
  Efficient LLM Inference over Long Sequences

  Python

  •

  Apache License 2.0

  •19•0•0•0•Updated Nov 29, 2024Nov 29, 2024

• LLM_Sizing_Guide

  Public
  A calculator to estimate the memory footprint, capacity, and latency on NVIDIA AMD Intel

  Python

  •10•0•0•0•Updated Nov 24, 2024Nov 24, 2024

• SageAttention-rocm

  Public
  ROCm Quantized Attention that achieves speedups of 2.1x and 2.7x compared to FlashAttention2 and xformers, respectively, without lossing end-to-end metrics across various models.

  Cuda

  •

  Apache License 2.0

  •181•4•0•0•Updated Nov 21, 2024Nov 21, 2024

• axolotl-amd

  Public
  Go ahead and axolotl questions

  Python

  •

  Apache License 2.0

  •1.1k•0•0•0•Updated Nov 16, 2024Nov 16, 2024

• skypilot

  Public
  SkyPilot: Run AI and batch jobs on any infra (Kubernetes or 12+ clouds). Get unified execution, cost savings, and high GPU availability via a simple interface.

  Python

  •

  Apache License 2.0

  •716•0•0•0•Updated Nov 7, 2024Nov 7, 2024

• flash-attention-docker

  Public
  This is a repository that contains a CI/CD that will try to compile docker images that already built flash attention into the image to facilitate quicker development and deployment of other frameworks.

  Shell

  •

  Apache License 2.0

  •0•1•0•0•Updated Oct 26, 2024Oct 26, 2024

• flash-attention-rocm

  Public
  ROCm Fork of Fast and memory-efficient exact attention (The idea of this branch is to hope to generate flash attention pypi package to be readily installed and used.

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •1.9k•0•0•0•Updated Oct 26, 2024Oct 26, 2024