request_success_total metric not shown after upgrade vllm 0.5.5 -> 0.6.2 and turn on num-scheduler-steps option #9526
yongho-chang
announced in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Hello recently i upgraded vllm 0.5.5 -> 0.6.2 and turn on num-scheduler-steps option.
after work.
below metric is not shown.
request_success_total
e2e_request_latency_seconds_sum
request_generation_tokens_sum
request_prompt_tokens_sum
is it bug?
is seems because of num-scheduler-steps option
my vllm options are bellow.
- --tensor-parallel-size
- "4"
- --load-format
- auto
- --max-model-len
- "8192"
- --block-size
- "16"
- --gpu-memory-utilization
- "0.95"
- --num-scheduler-steps
- "8"
- --uvicorn-log-level
- warning
- --disable-log-requests
Beta Was this translation helpful? Give feedback.
All reactions