How to free up memeory?

[popconst]

Usecase:
I am using this repo to summarize a large list of documents one by one.

Problem:
After a certain amount I am running out of memory on my Macbook M1.

Assumption:

• There is a way to clear up the chat history so that the model does not use as much memory.

Is this possible or the right approach?

[giladgd]

@popconst Whenever there's no reference to an instance of LlamaContext or LlamaModel the garbage collector will prune the memory used by these objects on the llama.cpp side, so try to get rid of all the references you don't need anymore before you create a new instance of LlamaContext or LlamaModel.

For example, if you have a global variable that holds a LlamaModel instance, it still counts as a reference, so try to set it to null.