When using roller, the speed is too slow

[tbaram2]

First of all, exllama v2 is a really great module.

But there is one problem.

In the past exllama v1, there was a slight slowdown when using Lora, but it was approximately 10%.

However, in the case of exllama v2, it is good to support Lora, but when using Lora, the token creation speed slows down by almost 2 times.
(13B based on 4090: without lora 80 tokens/s => with lora 30 tokens/s)

Does anyone know how to solve this?

[turboderp]

Are you using the same LoRA in both cases?

[tbaram2]

in oobabooga.

--model TheBloke_MythoMax-L2-13B-GPTQ_gptq-4bit-32g-actorder_True
--lora nRuaif_BluemoonRP-L2-13B-This-time-will-be-better

with same model and lora in exllama v1, it was approximately 10% slow. but exllamav2 almost 60%~70% slow.

Thanks.