Context management- implementing --keep.

[mseddon]

In llama.cpp, the main command supports saving the prompt via the --keep flag, which is handy for long running chat sessions. Is there a way to simulate this with the current API? After a few exchanges, it appears the context fills up and the whole system goes bananas.

What I really need is access to llama_kv_cache_seq_shift and llama_kv_cache_seq_rm

[giladgd]

@mseddon I'm currently working on #85, which will include better support for infinite test generation and automatic context swapping.
I'll ensure the swap size is also configurable (by default, it'll be automatic and dynamic depending on the prompt).

I aim to make the API as high-level as possible with good defaults, so while you can still configure more advanced parameters, using this library won't require too much technical understanding of the underlying code.
In other words, I'd like to make this library smart enough so you wouldn't need or even want to directly use llama_kv_cache_seq_shift or llama_kv_cache_seq_rm