Chat history/memory

[gurbindersingh]

Does Llama.cpp have some built-in way to handle chat history in a way that the model can refer back to information from previous messages? Without simply sending the chat history as part of the prompt, I mean. In a similar way ChatGPT seems to be able to. I've tried using the --prompt-cache and --prompt-cache-all options but it seems to overrides the the cache file with each new prompt. I am not sure these even do what I need.

[gurbindersingh]

I did not manage to get this work. But it seems like the go-to way to implement an LLM memory is to use something like described in the Langchain documentation. And this article on pineconde.io I found.

TLDR

• Pass the last k messages along with the prompt
• Create a summary of the past conversation and pass this along with the prompt
• Use a combination of the previous two
• Use a vector database.

I hope this helps anyone else that was confused about this.

[metaskills]

Another alternate answer is to use Zep which works with any inference SDK, framework, or language. https://www.unremarkable.ai/llama-chat-history-with-zeps-ai-memory-knowledge-graph/