Is it possible to fine tune the GGUF model?

[yuneun92]

Can I fine tune the GGUF model? Or do I need to fine tune with the foundation model and then quantize it again?
If we can fine tune the GGUF itself, could you please let me know how, or give me the related discussion link? Thank you so much!

[hunt-47]

you can finetune llama based gguf models using llama.cpp

[yuneun92]

Thank you for replying :) Have a nice day!

[0xgigabyte]

Hey, Could you share a tutorial or article on steps to train it

[dspasyuk]

@hunt-47 Interesting, how did I miss that part in llamacpp? I normally use LllamaFactory and Lora to finetune the base model. It still needs an A100 GPU with 80 GB of VRAM to do that. Could you share the specific command you use to finetune gguf model?