[User] Insert summary of your issue or enhancement..

[Lzhang-hub]

Expected Behavior

For a prompt will generate 1000 output tokens, is there function to stop generate token manualy?

[shibe2]

How do you start generation?

[Lzhang-hub]

I run with command: ./server -m model/ggml-model.gguf -c 2048, and then post request.

[shibe2]

With streaming (stream: true in the request), when you received enough text, close the connection.