Skip to content

Commit 5b6fe23

Browse files
[Bugfix][Benchmark] Make sure the output length > 0 when testing prefill workload. (#20786)
Signed-off-by: KuntaiDu <kuntai@uchicago.edu> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
1 parent f0c98ca commit 5b6fe23

File tree

1 file changed

+3
-0
lines changed

1 file changed

+3
-0
lines changed

benchmarks/benchmark_dataset.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -324,6 +324,9 @@ def sample(
324324
input_low = int(real_input_len * (1 - range_ratio))
325325
input_high = int(real_input_len * (1 + range_ratio))
326326
output_low = int(output_len * (1 - range_ratio))
327+
# Ensure the lower bound for output length is at least 1 to prevent
328+
# sampling 0 tokens, which can cause request failures.
329+
output_low = max(output_low, 1)
327330
output_high = int(output_len * (1 + range_ratio))
328331

329332
# Add logging for debugging

0 commit comments

Comments
 (0)