why does ./main use vram even if i did -ngl 0 -ngld 0? #3587
-
|
how to make it only use cpu ram? i was running 2 instance. anyone knows how to limit to running in cpu ram only? |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 4 replies
-
|
Prompt processing can use VRAM even when not offloading layers. As far as I know, there isn't a way to disable that, so basically the solution would be to compile an executable that only has CPU support and use that in the case you want to be use it won't touch the GPU. Just note that ingesting the prompt/running stuff like |
Beta Was this translation helpful? Give feedback.
-
|
@KerfuffleV2 make sense to compile another executable. it's strange but thx. i'm doing so now. if it works, will close this discussion. |
Beta Was this translation helpful? Give feedback.
-
|
I'll be pretty surprised if it uses VRAM when compiled without GPU support. :) |
Beta Was this translation helpful? Give feedback.
-
|
@KerfuffleV2 you are a genius! another pure cpu compiled instance work. by the way, possible to fix this? :D much prefer to keep 1 instance instead. |
Beta Was this translation helpful? Give feedback.
-
|
@KerfuffleV2 on second thoughts, i can live with this. |
Beta Was this translation helpful? Give feedback.
Prompt processing can use VRAM even when not offloading layers. As far as I know, there isn't a way to disable that, so basically the solution would be to compile an executable that only has CPU support and use that in the case you want to be use it won't touch the GPU.
Just note that ingesting the prompt/running stuff like
perplexityis much, much slower on CPU.