Llama.cpp server stopping

[appvoid]

I'm trying to stop ./server inference on the middle of the process without any good results so far.

I'm stuck here. Can someone explain me better how can I "stop" the the process when requesting a streamed inference on the server?

I'm using javascript's Fetch.

[jhen0409]

The server example use fetch & AbortController, it should work in the token generating. However the prompt processing is usually one-run, so it may not be interrupted immediately unless n_batch is set to a small value.