Using Nvidia GPU on Windows for running models

[sussyboiiii]

Hello,
the title basically describes my problem. I'm on windows, I have installed CUDA and when trying to make with cuBLAS it says your not on linux and then stops making. Is there just no GPU support for Windows or am I missing something?
Thank you!

[bullno1]

There is. I just built it yesterday.

You should use cmake to generate a msvc project. nvcc on windows only works with cl (MSVC compiler), not clang.

I guess ninja would work too.

Took me a bit longer than I want to admit to just to get this: https://github.com/bullno1/hey/blob/master/msvc.bat

[sussyboiiii]

Thank you!
So it's not properly supported at the moment? Because I've been using make to build in the past.

[bullno1]

It is properly supported.

[Anton-V-K]

I've checked the build on Windows recently with CUDA - it goes quite good:

• VS2022 17.7.6
• CMake 3.28.1
• CUDA 12.3.1

> mkdir build
> cd build
> cmake .. -DLLAMA_CUBLAS=ON
> cmake --build . --config Release

[sussyboiiii]

Thanks!