Dynamic Model Loading with Docker Deployment #8077
ArijitSinghEDA
announced in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
If I use the Dockerfile to deploy a vLLM server, does it only support single model deployment, or can I load models dynamically as well?
The Dockerfile contains only this line as its execution:
ENTRYPOINT ["python", "-m", "vllm.entrypoints.openai.api_server"]
In case it does support loading multiple model loading, how do I use it then?
Beta Was this translation helpful? Give feedback.
All reactions