How to recover from cuda Out of memory

[c0ffymachyne]

Hi,

I would like to thank you all for llama.cpp !! It’s great. I am new to llama.cpp and have just recently integrated into my cpp program and am running into an issue. The application work great but I cannot seem to be able to recover it from OOM error. I am using cuda compiled version with gpu offloading. When my application runs into OOM and even though I reload the model it still is causing segmentation fault next time I try to run inference. Is there any way to recover from OOM ? I did notice majority of examples exit application in case of such failure but can I recover from OOM without restarting whole application ?