Lightweight web interface ?

[alexandre-romain]

Got LLAMA + the 65b models running (slowly, but usable) on a Debian 11 Hyper-V VM running on a dual Xeon 5218R HP Proliant DL385 server (SSD disks, 128Go RAM)

I would like to be able to use it through a 'ChatGPT-like' interface but didn't find any 'out of the box' solution that doesn't require tons of dependencies ...

Anything you could link me to ?

Thanks a lot !

[ido-pluto]

You can check out CatAI. It is very simple to install.

It also has a chatGPT like ui.
checkout the additional commands for that.

[nuance1979]

You can also try llama-server.

[SlyEcho]

Or koboldcpp