CPU support on the docker image?

[dimknaf]

🚀 The feature, motivation and pitch

I have been trying to use the docker image and the pipeline with CPU support.
Has anyone tried it? What is the best way to go.

Alternatives

No response

Additional context

No response

[jakep-allenai]

This would be such a valuable contribution! We haven't tried it yet, but our Dockerfile basically mirrors the VLLM one: https://github.com/vllm-project/vllm/blob/main/docker/Dockerfile

And they have this one: https://github.com/vllm-project/vllm/blob/main/docker/Dockerfile.cpu

If you get anything working, please let us know

[dimknaf]

I managed to do it with tricks and pointing to an external vLLM, and happy to see that you are planning to implement something like this