Performance degradation when quantizing QLoRA finetunned model

[TomekPro]

Hi, I wanted to convert a QLoRA fine-tuned model to GGUF for CPU inference but unfortunately I observe a big model performance degradation (accuracy).
QLoRA -> GOOD
QLoRA -> merge adapter after dequantization -> GOOD
QLoRA -> merge adapter after dequantization -> convert to GGUF with llama.cpp -> BAD
QLoRA -> merge adapter after dequantization -> quantized again with the same bnb config -> BAD

From what I've read here https://kaitchup.substack.com/p/training-loading-and-merging-qdora it should work with AWQ quantization but this is not supported here. Maybe it is worth adding, I saw there was an attempt a few months ago or maybe I should use some different quantization parameters. I did python convert-hf-to-gguf.py <MERGED_MODEL_PATH> --outtype q8_0 --outfile <OUTPUT_MODEL_NAME.gguf>
Any idea what could be a solution here?

Thanks
Tomek