is llama.cpp now faster than exllamav2? i just ran a sample and it seems so. is this how it is now?

[sprappcom]

llama.cpp now the fastest way to run llms?

[slaren]

Are you using flash attention with exllama?

[sprappcom]

nope.

do u know how to run the fastest llama ever? like the fastest way to run llama.
i've checked decillm, (i cant find a way to run together ai locally).

any ideas?

[slaren]

From what I understand, exllama is significantly faster when used with flash attention.

[sprappcom]

i tried exllama. it's faster by 10-15% not sure if flash attention is used.