LocalAI: OpenAI API compatible based on llama.cpp and ggml

[mudler]

Hi 👋

First of all, as always, a big thanks to @ggerganov for the impressive work - I will never stop saying it.

I've created https://github.com/go-skynet/LocalAI and thought to share it here. It's a Self-hosted, community-driven simple local OpenAI-compatible API written in go. It can be used as a drop-in replacement for OpenAI. The scope is to use code bindings to create a generic API that runs ggml's supported model efficiently (including GPT4ALL, or StableLM) under the same API umbrella without friction from the user (since there are many llama.cpp fork/based code, I sensed the need to make them in a single, convenient place for the user).

It is already having big impact on different projects that are based so far on OpenAI APIs, for instance for Kubernetes cluster analysis: https://medium.com/@tyler_97636/k8sgpt-localai-unlock-kubernetes-superpowers-for-free-584790de9b65

Here is the Github repo: https://github.com/go-skynet/LocalAI

[x4080]

Hi @mudler , I was looking for efficient API to connect to llama.cpp, is your API still make llama.cpp responsive ?

Thanks, and love the work

[mudler]

I'm not sure what do you mean by "llama.cpp responsive", but it uses llama.cpp behind the scenes, with the go binding here which are updated constantly: https://github.com/go-skynet/go-llama.cpp

[x4080]

@mudler Cool, I'll try it