Stop and Continue response like llmstudio #8969
Unanswered
electro199
asked this question in
Q&A
Replies: 2 comments
-
the same question. |
Beta Was this translation helpful? Give feedback.
0 replies
-
In llama-server, to stop a request, just close the connection. To continue it, just resent the whole prompt (so it doesn't work with chat endpoint, eh) with the partially generated message you saved |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
In LLM studio there is option to stop generation edit the response and then continue so LLM use the provided text/token for completion. How to replicate that in the llama-server ?
Beta Was this translation helpful? Give feedback.
All reactions