How to fine-tune a .gguf model?

[SeanZhang7]

May I ask How could I fine-tune a .gguf model? I saw some discussions about converting it into pytorch form. But this method seems not infeasible.

[chrisalbertson]

Total beginner here but, it seems to me what you do is apply an LoRA adaper to the .gguf file and llama.cpp does the work of applying it to the model in real time.

Can you then save the adapted model? I've not figured that out yet.

[SeanZhang7]

I convert a model from HF to .gguf, and I try to fine-tune that gguf model. I haven't figured out hoe to do that.
Do you mean I can fine-tune it by apply a LoRA adaper to the .gguf file?