Why session_tokens and KV_cache are not matched when max context is reached?

[Zapotecatl]

Hi, I understand that by setting: "--prompt-cache = C:\MyConversation.bin...", "--prompt-cache-all" and "--keep = -1", I could recover the context and previous conversations.

However, I can't understand when the maximum context is reached and it is necessary to make room for new conversations, the session tokens stop saving instead of matching with the kv cache.

I understand this:

if (n_past + (int) embd.size() >= n_ctx) {

  // [...]

  const int n_left    = n_past - params.n_keep;
  const int n_discard = n_left/2;

  llama_kv_cache_seq_rm (ctx, 0, params.n_keep, params.n_keep + n_discard);
  llama_kv_cache_seq_add(ctx, 0, params.n_keep + n_discard, n_past, -n_discard);

  n_past -= n_discard;

  path_session.clear(); // Why was it decided to stop saving the entire subsequent conversation?

  /*
 Why aren't the session tokens matched accordingly so that we can continue storing the conversation?
 Something like this (not considering the case of n_keep = -1)

 auto first = session_tokens.begin() + params.n_keep;
 auto last = session_tokens.begin() + (params.n_keep + n_discard);
 session_tokens.erase(first, last)
  */

}

Why are session tokens not updated as expected (session_tokens.erase(first, last))?

Thanks in advance!

[ggerganov]

Seems like it has been overlooked and a fix is necessary.