Skip to content

Issues: neuralmagic/guidellm

Beta
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

34 Open 29 Closed
34 Open 29 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Standardized Sweep Numbers priority-medium
#156 opened May 8, 2025 by rgreenberg1
Sweep Mode Improvements - get closer to approaching peak-throughput load-request load-request workstream priority-high
#155 opened May 8, 2025 by rgreenberg1
Standardize dataset profiles for reasoning benchmarking priority-high
#154 opened May 8, 2025 by rgreenberg1
Support Audio to text multi-modal benchmarking in GuideLLM priority-high
#153 opened May 8, 2025 by rgreenberg1
1
Support VL Model Benchmarking multi-modal Support for benchmarking new multi-modal models priority-high
#152 opened May 8, 2025 by rgreenberg1
Event loop is closed error. bug Something isn't working priority-high
#147 opened May 5, 2025 by phalani-paladugu
@markurtz
1
support for multi-turn qa dataset priority-medium
#138 opened Apr 25, 2025 by ashishkamra
GuideLLM Sweep Benchmark: Low KV Cache Utilization and Limited QPS on H100 GPUs
#136 opened Apr 25, 2025 by kaushikmitr
4
GuideLLM DS Generation Engine #26
#134 opened Apr 24, 2025 by SharonGil
1
Add Docker and Kubernetes Support for GuideLLM Benchmarking
#119 opened Apr 17, 2025 by wangchen615
1
Sweep mode with --max-seconds fails ungracefully if calculated RPS is too low priority-low
#115 opened Apr 15, 2025 by sjmonson
@sjmonson
[Bug] Possibly incorrect output dump ? (The actual data appears to be fine) good first issue Good for newcomers priority-medium
#114 opened Apr 15, 2025 by thameem-abbas
1
[FeatureReuqest] Auto-sweep with model saturation detection enhancement New feature or request
#108 opened Apr 8, 2025 by markVaykhansky
1
[Feature Request] TTFT & ITL Timeouts priority-low
#107 opened Apr 8, 2025 by markVaykhansky
[FeatureRequest] Constraint input & output sizes no matter the dataset
#106 opened Apr 8, 2025 by markVaykhansky
1
[Feature Request] Max error rate parameter priority-low
#105 opened Apr 8, 2025 by markVaykhansky
[Feature Request] Testing with defined prefix lengths priority-high
#104 opened Apr 4, 2025 by thameem-abbas
Enable Batch Inferencing Benchmarking Support priority-medium
#102 opened Mar 31, 2025 by rgreenberg1
Automatic performance stabilizization - so no need to set max-requests or max-seconds analysis analysis workstream - relating to adding in features to do analysis on extrapolating results automation automation workstream priority-high
#101 opened Mar 31, 2025 by rgreenberg1
@markurtz
Land GuideLLM UI in Upstream Repository priority-high UI Front-end workstream
#98 opened Mar 12, 2025 by rgreenberg1
@DaltheCow
INFO log level is to verbose enhancement New feature or request good first issue Good for newcomers priority-low
#97 opened Mar 11, 2025 by sjmonson GuideLLM v0.2.0 - CI/CD Finalization, Documentation Expansion, and Backend Support
@sjmonson
Have GuideLLM kick off a vLLM server automatically to avoid having the user install vLLM and assign the target themselves enhancement New feature or request
#95 opened Mar 9, 2025 by rgreenberg1
@markurtz @rgreenberg1
1
Max RPS in sweep mode could be higher
#93 opened Mar 7, 2025 by dagrayvid
5
Integrate lm-eval into GuideLLM accuracy-eval accuracy-evaluation workstream
#90 opened Mar 6, 2025 by rgreenberg1
@jennyyangyi-magic
1
Enable vLLM scheduler specific Metrics metrics Metrics workstream
#86 opened Mar 4, 2025 by rgreenberg1
@markurtz @rgreenberg1
ProTip! Mix and match filters to narrow down what you’re looking for.