bzhng-development
Popular repositories Loading
-
sglang
sglang PublicForked from sgl-project/sglang
SGLang is a fast serving framework for large language models and vision language models.
Python
-
vllm
vllm PublicForked from vllm-project/vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
-
cube
cube PublicForked from Roblox/cube
Roblox Foundation Model for 3D Intelligence
Jupyter Notebook
-
flexible-inference-bench
flexible-inference-bench PublicForked from CentML/flexible-inference-bench
A modular, extensible LLM inference benchmarking framework that supports multiple benchmarking frameworks and paradigms.
Python
-
BentoML
BentoML PublicForked from bentoml/BentoML
The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
Python
-
flashinfer
flashinfer PublicForked from flashinfer-ai/flashinfer
FlashInfer: Kernel Library for LLM Serving
Cuda
Repositories
- sglang Public Forked from sgl-project/sglang
SGLang is a fast serving framework for large language models and vision language models.
bzhng-development/sglang’s past year of commit activity - TensorRT-LLM Public Forked from NVIDIA/TensorRT-LLM
TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and support state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in performant way.
bzhng-development/TensorRT-LLM’s past year of commit activity - TransformerEngine Public Forked from NVIDIA/TransformerEngine
A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit floating point (FP8) precision on Hopper, Ada and Blackwell GPUs, to provide better performance with lower memory utilization in both training and inference.
bzhng-development/TransformerEngine’s past year of commit activity - vllm Public Forked from vllm-project/vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
bzhng-development/vllm’s past year of commit activity - NeMo-Skills Public Forked from NVIDIA/NeMo-Skills
A project to improve skills of large language models
bzhng-development/NeMo-Skills’s past year of commit activity - triton Public Forked from triton-lang/triton
Development repository for the Triton language and compiler
bzhng-development/triton’s past year of commit activity - SpecForge Public Forked from sgl-project/SpecForge
Train speculative decoding models effortlessly and port them smoothly to SGLang serving.
bzhng-development/SpecForge’s past year of commit activity
People
This organization has no public members. You must be a member to see who’s a part of this organization.
Top languages
Loading…
Most used topics
Loading…