gguf_init_from_file: tensor 'token_embd.weight' number of elements (131072000) is not a multiple of block size (0)

[douglasdew]

In support of my goal of learning this codebase by stepping through it in Xcode's debugger and profiling it with Instruments, I have created a new Xcode project and added to that project the nineteen .h/.c/.cpp/.m files that are necessary to complete a build of the "main" example.

I'm encountering a model loading error when attempting to run the code:

gguf_init_from_file: tensor 'token_embd.weight' number of elements (131072000) is not a multiple of block size (0)

That error occurs when attempting to load the llama-2-7b-chat.Q5_K_M.gguf file.

I'm able to use make to successfully build the llama.cpp repo code and to run the "main" example using the llama-2-7b-chat.Q5_K_M.gguf file. So, I assume that I've caused a problem when creating my Xcode project.

I'm debugging now, trying to determine what sort of problem that I've caused. And, I'm looking through the Makefile to see if there were any settings in there which I failed to account for in my Xcode build.

I'd be grateful for any suggestions.

[douglasdew]

Also, in order to make the compiler happy in Xcode, I had to comment out all uses of "release" in the ggml-metal.m file. The "release" function is not available when using ARC. There were no complaints about those usages of "release" when I built from the command line using make, so that suggests to me that there are some build settings in my Xcode project which might be incorrect.

[douglasdew]

I see in the Makefile that LLAMA_METAL should be defined in my Xcode build settings. I'm looking for other settings which I've missed.

[douglasdew]

The Makefile very helpfully prints out all of the settings. Thank you very much to whomever added the print-out lines to the Makefile.

[douglasdew]

I got everything working in Xcode without having to make any code changes. Thank you very much for providing code that is easy to understand.

[paulocoutinhox]

I have this same error:

gguf_init_from_file: tensor 'token_embd.weight' number of elements (131137536) is not a multiple of block size (0)

#3724

Whats is your solution?

Thanks.