Not able to start server with swap mode #5050
sathyanarays
announced in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
By default, if I start the server using
python -m vllm.entrypoints.openai.api_server
, it starts the server with 'recompute' preemption mode. I don't see any option to configure the scheduler to use CPU swap space. Is this something needs to be implemented? Or is it available and I am not able to find the right documentation?Beta Was this translation helpful? Give feedback.
All reactions