Running on specific GPU

[rami2102]

How can I run llama cpp on specific GPU (I have few GPUs on my PC), both for main and server executables?

[slaren]

There are two ways to do this:

• Use -sm none -mg <gpu> in the command line. -sm none disables multi GPU and -mg selects the GPU to use.
• Set the CUDA_VISIBLE_DEVICES environment variable to the GPU that you want to use

In my experience, setting CUDA_VISIBLE_DEVICES results in slightly better performance, but the difference should be minor.