Simple API Wrapper

[montecarlo26]

A wrapper of llamacpp could trivialize instantiating and using a llm directly in C++. I'm referring to something like this:

#include "api_wrapper.h"

void main() {

	language_model the_language_model;
        the_language_model.params.model = "C:/Users/Quill/Documents/ai/models/mistral-7b-v0.1.Q8_0.gguf";
        the_language_model.initialize();
        std::string input_text = "Hi, how are you?";
        std::string output_text;
        the_language_model.process_text(input_text, output_text);
        printf("model response: %s\n", output_text.c_str());

}

I have a small example project here which I don't really plan to update at all, just a proof of concept. Anyways, an API wrapper would be nice to have. I don't think such a thing exists, at least that I was able to find.

[wilderfield]

Not a bad idea.

You could even add Python bindings.

nanobind or pybind11.

[ngxson]

Maybe related to #5215