working example of vllm using whisper?

[silvacarl2]

does anyone have a tiny working example of of vllm using whisper?

[jeejeelee]

See: https://github.com/vllm-project/vllm/blob/main/examples/offline_inference/whisper.py

[silvacarl2]

yes we tried that. i dont think that whisper is ready yet on vllm. we installed on an A40 and it still says there is not enough memory. thats not possible.

[jeejeelee]

Could you plz provide your env info?