Not able to start local llama Server

[sorohere]

I am unable to activate the llama server on my mac. I tried running .\main -m llama-2-7b.Q4_0.gguf -n 128 on Google Colab and it worked well there, but now I am facing issues on my local machine. how run for macOS ?

[NikhilKalloli]

You can try this: ./server -m /path-to-the-model/gemma-2b-Q4_0.gguf -ngl 999 -c 2048.
It specifies the model path and sets the number of GPU layers to 999 and the context size to 2048.
Ensure you replace /path-to-the-model/ with the actual path where your model is stored.