KeyError: 'rms_norm_eps' during convert?

[bioshazard]

Anyone have any guidance about making models like this work in llama.cpp? https://huggingface.co/WhereIsAI/UAE-Large-V1/

Ran into this error attempting to use convert.py. Never attempted a GGUF before. Thanks!

llama.cpp> python .\convert.py ..\UAE-Large-V1\       
Loading model file ..\UAE-Large-V1\model.safetensors
Traceback (most recent call last):
  File "C:\Users\joe\Documents\root\workspaces\app-dev\llama.cpp\convert.py", line 1279, in <module>
    main()
  File "C:\Users\joe\Documents\root\workspaces\app-dev\llama.cpp\convert.py", line 1218, in main
    params = Params.load(model_plus)
             ^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\joe\Documents\root\workspaces\app-dev\llama.cpp\convert.py", line 318, in load
    params = Params.loadHFTransformerJson(model_plus.model, hf_config_path)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\joe\Documents\root\workspaces\app-dev\llama.cpp\convert.py", line 257, in loadHFTransformerJson
    f_norm_eps        = config["rms_norm_eps"],
                        ~~~~~~^^^^^^^^^^^^^^^^
KeyError: 'rms_norm_eps'

[bioshazard]

I found an hf convert script in the project and I see bert models are not supported: NotImplementedError: Architecture "BertModel" not supported!. So.. nevermind! Thanks for llama.cpp <3

[Maruthi-Tally]

Hello, Did anyone got to know how to overcome this issue ?

[Pierrelouis2]

hello,
Maybe this issue can help you #5009

[iamlemec]

This works now! You need to use convert-hf-to-gguf.py instead, so something like

python convert-hf-to-gguf.py UAE-Large-V1 --outfile models/uae-large-v1-f16.gguf --outtype f16

where UAE-Large-V1 should be a directory containing the huggingface repo for the model, which you can download locally with git or huggingface-cli. After that you can try it out with the embedding example.

[oldgithubman]

This works now! You need to use convert-hf-to-gguf.py instead, so something like

python convert-hf-to-gguf.py UAE-Large-V1 --outfile models/uae-large-v1-f16.gguf --outtype f16

where UAE-Large-V1 should be a directory containing the huggingface repo for the model, which you can download locally with git or huggingface-cli. After that you can try it out with the embedding example.

This should be added to the main readme