i just bought a laptop, rtx 3060 running docker llama.cpp:full-cuda with CUDA error 35 at ggml-cuda.cu:5509: CUDA driver version is insufficient for CUDA runtime version

[hiqsociety]

i just bought a laptop, rtx 3060 running docker llama.cpp:full-cuda with CUDA error 35 at ggml-cuda.cu:5509: CUDA driver version is insufficient for CUDA runtime version
current device: 21996

how to resolve this?

im using gguf airoboros 13b from thebloke huggingface

[hiqsociety]

i am using 11.5.119 on my host but the full cuda dockerfile is using 11.7

how to resolve?

[staviq]

Check this link, if you scroll down, there is a table including cuda 11, specifying minimal driver version for given cuda version: https://docs.nvidia.com/cuda/cuda-toolkit-release-notes/index.html

You likely simply need to update the drivers.

[hiqsociety]

@staviq thanks! i stayed up all night trying to "fix" this. but now the problem is, i realised the driver is actually 12.2
i tried to change the dockerfile to 12.2 and it says manifest not found. so how to resolve?

[hiqsociety]

@staviq used "non-docker" version.
got it working finally. will figure out the docker one in future. thx