is convert pt to gguf possible by llama.cpp? #8773
Replies: 2 comments
-
Hey, did you find a solution for it? I want to convert the PT model to GGUF in order to use it in Ollama, but I don't know how to do it. This is the model I want to try |
Beta Was this translation helpful? Give feedback.
-
I haven't tried conversion from .pt or .pth myself, but a quick search turned up these: But really, if you only want to run the models, you're much better off just finding a ready-made GGUF. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
Hi, everyone. I am new to the llama.cpp community. Recently, I tried to use llama.cpp to load some models I found. During this process, I encountered a serious problem: the format conversion among different storage formats of LLM is quite messy.
For example, if I have a .pt/.pth file, should I first convert it to safetensor and then convert that safetensor file to gguf? Additionally, I have found very few methods or all-in-one tools/tutorials to handle these conversions effectively.
Moreover, I have discovered another issue: not all .pth/.pt files are the same. This raises the question of which specific files are necessary for a successful conversion. I have tried some repositories[1][2][3] that convert .pt files to safetensor, but they only produce a single safetensor file, which does not include config.json and other potentially necessary files.
I apologize for my unclear expression. These problems seem to be intertwined, and I am struggling to explain them clearly. If anyone has experience with these issues or can provide some guidance, I would greatly appreciate it.
[1] https://github.com/jtabox/safetensors-converter/blob/main/safetensors_converter.py
[2] https://github.com/Silver267/pytorch-to-safetensor-converter
[3] https://github.com/huggingface/safetensors/blob/main/bindings/python/convert.py
Beta Was this translation helpful? Give feedback.
All reactions