I would like to remove tasks from the queue if the associated request is terminated (server)

[redlion0929]

Hello, I have an inquiry for yours.
Currently, I am hosting an embedding server using server.
I've come across an issue during use and decided to reach out for help.
When a client sends a request and subsequently terminates it, tasks already in the queue continue to be processed without recognizing the termination.
If a client makes a massive number of requests and then disconnects, the server could become paralyzed.
Is there a way to modify this behavior within llama.cpp?
TLDR; I would like to remove tasks from the queue if the associated request is terminated.

[phymbert]

Good idea, please create an issue, ideally at each server loop, task cancellation should be checked.