[Bug] New opea/vllm:latest image doesn't work on kubernetes

[yongfengdu]

Priority

P2-High

OS type

Ubuntu

Hardware type

Xeon-GNR

Installation method

• Pull docker images from hub.docker.com
• Build docker images from source
• Other

Deploy method

• Docker
• Docker Compose
• Kubernetes Helm Charts
• Other

Running nodes

Single Node

What's the version?

opea/vllm:latest

Description

New vllm:latest doesn't work on kubernetes.
It failed to start due to:
Error: failed to create containerd task: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "python3": executable file not found in $PATH

kubernetes is running vllm with non-root user(In this case uid=1000) for security reason.
In the new vllm:latest image, the python is pointing to a directory under /root, which non-root user doesn't have permission:
root@a3ee2a23744d:/workspace# ls -l /opt/venv/bin/python
lrwxrwxrwx 1 root root 75 Apr 9 05:30 /opt/venv/bin/python -> /root/.local/share/uv/python/cpython-3.12.9-linux-x86_64-gnu/bin/python3.12

Reproduce steps

cd GenAIInfra/helm-charts/common
helm dependency update vllm
helm install myvllm vllm -f cpu-values.yaml

Raw log

Attachments

No response

[chensuyue]

The opea/vllm:latest build with https://github.com/vllm-project/vllm/blob/v0.8.3/docker/Dockerfile.cpu, need to submit an issue to vllm.

[yongfengdu]

File issue on vllm project to track:
vllm-project/vllm#16402

[joshuayao]

Hi @yongfengdu, the issue should be fixed by vllm-project/vllm#16454. Could you please verify if we can close this issue?

[yongfengdu]

Yes, I've verified the upstream has fixed this issue in 0.8.4 vllm release.
I think it's better to keep this Issue open till opea/vllm upgraded to 0.8.4 or later? (Now it's 0.8.3 based)

[joshuayao]

Yes, I've verified the upstream has fixed this issue in 0.8.4 vllm release. I think it's better to keep this Issue open till opea/vllm upgraded to 0.8.4 or later? (Now it's 0.8.3 based)

Yes, we will try vllm v0.8.4.

@chensuyue could you please help try vllm v0.8.4? Thanks.

[yongfengdu]

Yes, I've verified the upstream has fixed this issue in 0.8.4 vllm release. I think it's better to keep this Issue open till opea/vllm upgraded to 0.8.4 or later? (Now it's 0.8.3 based)

Yes, we will try vllm v0.8.4.

@chensuyue could you please help try vllm v0.8.4? Thanks.

We have committed a workaround for this issue in helm chart, so this does not impact any functionality and is not urgent for 1.3 release.
It's up to you to evaluate whether it's proper to use 0.8.4 or 0.8.3 based vllm, considering the test coverage and 1.3 release date.