BentoML

All

111 repositories

llm-inference-handbook
Public
Everything you need to know about LLM inference
inference-optimization llm llm-inference inference-handbook inference-infrastructure
TypeScript
•
MIT License
•21•232•3•0•Updated Sep 19, 2025Sep 19, 2025
BentoML
Public
The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
python machine-learning deep-learning model-serving multimodal mlops ml-engineering ai-inference llm generative-ai
Python
•
Apache License 2.0
•872•8.1k•135•9•Updated Sep 19, 2025Sep 19, 2025
BentoYolo
Public
BentoML service of YOLO v8
Python
•
Apache License 2.0
•3•13•1•0•Updated Sep 17, 2025Sep 17, 2025
OpenLLM
Public
Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.
llama mistral fine-tuning mlops bentoml vicuna llm model-inference llmops llm-serving
Python
•
Apache License 2.0
•772•12k•3•2•Updated Sep 15, 2025Sep 15, 2025
yatai-common
Public
Go
•6•1•0•2•Updated Sep 15, 2025Sep 15, 2025
BentoVLLM
Public
Self-host LLMs with vLLM and BentoML
Jinja
•
Apache License 2.0
•18•149•2•0•Updated Sep 12, 2025Sep 12, 2025
llm-optimizer
Public
Benchmark and optimize LLM inference across frameworks with ease
Python
•
Apache License 2.0
•7•94•5•0•Updated Sep 12, 2025Sep 12, 2025
bentocloud-homepage-news
Public
2•3•0•0•Updated Sep 9, 2025Sep 9, 2025
helm-charts
Public
0•3•0•0•Updated Sep 3, 2025Sep 3, 2025
yatai-image-builder
Public
🐳 Build OCI images for Bentos in k8s
docker docker-image container container-image kubernetes k8s
Go
•12•19•6•12•Updated Sep 3, 2025Sep 3, 2025
comfy-pack
Public
A comprehensive toolkit for reliably locking, packing and deploying environments for ComfyUI workflows.
ai-art comfyui genai comfyui-manager comfyui-workflow comfyui-nodes comfyui-api
Python
•
Apache License 2.0
•25•179•10•3•Updated Aug 29, 2025Aug 29, 2025
kantoku
Public
A Process & Socket Manager built with zmq
Python
•
Other
•259•1•0•0•Updated Aug 28, 2025Aug 28, 2025
byoc
Public
Shell
•1•1•0•2•Updated Aug 28, 2025Aug 28, 2025
BentoProphet
Public
BentoML with Facebook Prophet
Python
•0•0•0•0•Updated Aug 27, 2025Aug 27, 2025
XGBoostDemo
Public
Python
•0•2•0•0•Updated Aug 19, 2025Aug 19, 2025
pipelines-workflows
Public
Gruntwork Pipelines for BentoML
20•0•0•0•Updated Aug 18, 2025Aug 18, 2025
asynq
Public
Simple, reliable, and efficient distributed task queue in Go
Go
•
MIT License
•853•0•0•0•Updated Aug 5, 2025Aug 5, 2025
BentoLlamaCpp
Public
BentoML + llama.cpp
Python
•0•0•0•1•Updated Jul 29, 2025Jul 29, 2025
BentoSGLang
Public
Python
•1•3•0•1•Updated Jul 29, 2025Jul 29, 2025
BentoSpacy
Public
Python
•0•0•0•1•Updated Jul 29, 2025Jul 29, 2025
BentoTriton
Public
Python
•
Apache License 2.0
•1•3•0•1•Updated Jul 29, 2025Jul 29, 2025
CLIP-API-service
Public
CLIP as a service - Embed image and sentences, object recognition, visual reasoning, image classification and reverse image search
cloud-native clip model-serving mlops ai-applications openai-clip model-inference model-inference-service
Jupyter Notebook
•
Apache License 2.0
•4•65•0•1•Updated Jul 29, 2025Jul 29, 2025
BentoLMDeploy
Public
Self-host LLMs with LMDeploy and BentoML
Python
•2•22•1•0•Updated Jul 7, 2025Jul 7, 2025
sentence-embedding-bento
Public archive
Sentence Embedding as a Service
Jupyter Notebook
•
Apache License 2.0
•3•15•0•1•Updated Jun 30, 2025Jun 30, 2025
Pneumonia-Detection-Demo
Public
Pneumonia Detection - Healthcare Imaging Application built with BentoML and fine-tuned Vision Transformer (ViT) model
computer-vision healthcare transformer health-care-application healthcare-imaging
Python
•2•8•1•1•Updated Jun 17, 2025Jun 17, 2025
BentoGradio
Public
Python
•0•0•0•0•Updated Jun 13, 2025Jun 13, 2025
BentoTinyCudaNN
Public
Python
•1•0•0•0•Updated Jun 12, 2025Jun 12, 2025
BentoResnetTensorFlow
Public
Python
•0•0•0•1•Updated May 30, 2025May 30, 2025
openllm-models
Public
Python
•16•51•0•0•Updated May 28, 2025May 28, 2025
bentocloud-cicd-example
Public
Python
•2•0•0•0•Updated May 20, 2025May 20, 2025