Tune DMMV_X and DMMV_Y in the Makefile

[manu-web]

Does anybody know how to set the values of DMMV_X and DMMV_Y in the Makefile for the best performance?

[dranger003]

Here are some resources:

https://github.com/ggerganov/llama.cpp/blob/master/docs/build.md#cuda
#1827 (comment)

These values determine how much data the GPU processes at once for the computationally most expensive operations and setting higher values is beneficial on fast GPUs (but make sure they are powers of 2). On my RTX 3090 setting LLAMA_CUDA_DMMV_X=64 LLAMA_CUDA_DMMV_Y=2 increases performance by 20%.

The variables have recently been renamed to GGML_CUDA_DMMV_X and GGML_CUDA_MMV_Y.