Running metal on older MacBook

[tarasovic7]

Hi,

I am completely new to this repository. I would like to run models on apple metal framework, but I always get an error.
When I run following code

./main -t 4 -m ../../models/yarn-llama-2-7b-64k.Q4_K_M.gguf --color -c 4096 --temp 0.7 --repeat_penalty 1.1 -n -1 -p "Write a story about llamas"
I am getting normal response. But running
./main -t 10 -ngl 32 -m ../../models/yarn-llama-2-7b-64k.Q4_K_M.gguf --color -c 4096 --temp 0.7 --repeat_penalty 1.1 -n -1 -p "Write a story about llamas"

Is producing following error.

ggml_metal_init: load pipeline error: Error Domain=CompilerError Code=2 "SC compilation failure There is a call to an undefined label" UserInfo={NSLocalizedDescription=SC compilation failure There is a call to an undefined label} llama_new_context_with_model: ggml_metal_init() failed llama_init_from_gpt_params: error: failed to create context with model '../../models/yarn-llama-2-7b-64k.Q4_K_M.gguf' main: error: unable to load model

I am getting model from following site:
https://huggingface.co/TheBloke/Yarn-Llama-2-7B-64K-GGUF/blob/main/README.md

Any Ideas what I am doing wrong?

[tarasovic7]

Problem solved by myself by installing missing phyton package. See #1730 (comment)

[sukualam]

maybe you still running on cpu? i try run with gpu still error

ggml_metal_init: load pipeline error: Error Domain=CompilerError Code=2

what your gpu?

[tarasovic7]

Just type pip install llama-cpp-python

[nospam2k]

Didn't work for me. Any other ideas? This is Sonoma with AMD RX 7700xt