Timeout parameter

[greg-gav]

Question is for 'queue-timeout' parameter: as I understand that is for requests that are queued waiting and waiting in queue, doesn't work for requests that are stuck in loop of reasoning which never finishes (happens for some Qwen models). For such cases where generating the response exceeds some timeout, can we have another parameter, say 'request-timeout' so it force stops inference after some set max time?

[cubist38]

Oh, that’s a great point @greg-gav — thank you for your recommendation! Would you be able to help me implement it? I’d really appreciate your support, as I’m currently quite busy with work, and it might take me a few days or even weeks to get to it on my own.