Concurrency Limit Issue at 32 with VLLM

[timeconnection]

I am currently leveraging CUDA 11.8 for accelerated inference, with VLLM version 0.3.3+cu118, deploying on an A800 GPU. The primary parameters in use include --gpu 1 --dtype float16 --tokenizer-mode slow --gpu-memory-utilization 0.6 --max-model 8096. However, I've encountered a concurrency limit capped at 32. Adjusting gpu-memory-utilization has not yielded improvements. How can I achieve higher concurrency levels