Running docker-image with CUDA-support failes with "./llama-cli: error while loading shared libraries:"

[itinance]

On my Ubuntu 22 machine with an RTX 4000, I have the nvidia cuda toolkit installed and the nvidia docker toolkit.

When I want to run the docker-image with CUDA-support:

docker run -v ./models:/models ghcr.io/ggerganov/llama.cpp:full-cuda --run -m models/LongWriter-llama3.1-8b-GGUF/LongWriter-llama3.1-8b-Q4_K_M.gguf -p "Building a website can be done in 10 simple steps:" -n 512
it shows an error:

`

./llama-cli: error while loading shared libraries: libcuda.so.1: cannot open shared object file: No such file or directory

`
Shouldn't this libcuda-file not be part of the docker-image? How can I make it work?

[itinance]

Okay, as pointed out on Reddit, the missing settings were:

--gpus all

and also: --n-gpu-layers 33 --ctx_size 512