gprc api

[sonic182]

There is any repo like "llama-server" or similar? so we can have a llama.cpp instance running a model and a GRPC api for sending prompts and receiving completions?

[ianscrivener]

Llama.cpp has a simple HTTP API server

There are a few other Bindings mentioned on the README.md homepage of this repo.

I use the Python API by Abetlen et all abetlen/llama-cpp-python.. which very conveniently mimics the OpenAI API... so you can use any of the OpenAI ChatGPT clients >> llama-cpp-python >> llama.cpp.

Sorry, I don't of a GRPC API for llama.cpp