Llama.cpp version used by llamafile

[Propfend]

I am trying to use llamafile interchangeably with llama.cpp, i come from the context of reverse proxying to llama.cpp servers, i thought at first llamafile woud work out of the box like some llama.cpp instance, but then there is a difference between current llama.cpp /slots endpoint and the actual one in llamafile. Its a good thing that it supports slots endpoint though. I havent found in any place on documentation about it, i guess this could be specified in the Technical aspects of the documentation. So i can have some means on my end to extend that to llamafile. take a look at Paddler. I am very happy to extend the project to Paddler and that would be such a feature.

Is there any will on support newer versions of llama.cpp? I know its very hard to keep up the pace with llama.cpp.

[cjpais]

The build is approx commit: a30b324 on llama.cpp

llamafile/llama.cpp/build-info.cpp

Line 2 in a9658c7

char const *LLAMA_COMMIT = "a30b324";

We are interested in pulling from upstream, but it may take some work

[reneleonhardt]

Will the 8 months of improvements and fixes (1,500 commits over 1,000 files) in upstream be integrated into llamafile at some point in the future?
ggml-org/llama.cpp@8b3befc...master
https://github.com/Mozilla-Ocho/llamafile/blob/main/llama.cpp/README.llamafile#L12

Would patches on top of git submodule / subtree make it easier to benefit from upstream advancements?

@cjpais I can't find that short hash in upstream.

$ git clone https://github.com/ggml-org/llama.cpp.git
$ cd llama.cpp
$ git log -1 a30b324
fatal: ambiguous argument 'a30b324': unknown revision or path not in the working tree.

[cjpais]

It would be nice to yes. I think we need to evaluate the changes and see what patches we have in place currently that enable llamafile to work with the codebase. But overall I think it would be a major benefit to the project. If you or anyone wants to help in this effort or start a discussion it would be much appreciated

[reneleonhardt]

I think it would be best, but I can't even find the mentioned base commit so it will be very hard to create a patch 😅

[cjpais]

Looks like i might have gotten it wrong earlier. I see an earlier sync to:

https://github.com/ggerganov/llama.cpp/8b3befc0e2ed8fb18b903735831496b8b0c80949

See: https://github.com/Mozilla-Ocho/llamafile/blob/main/llama.cpp/README.llamafile

[BradHutchings]

You might want to take a look at llama-server-one ("LS1"):

https://github.com/BradHutchings/llama-server-one

While I still build and will continue to build llamafiles for some of my clients, I recently took on the task of building something like llamafile, but with the latest llama.cpp. I sync the latest commits a few times per week.

LS1 got SSL support this weekend. It's pretty nifty. I hope you'll take a look and contribute there and here.

-Brad

[Propfend]

Nice! I am going to take a look soon, thanks!