How can I reuse the context kv cache from last request in llama.cpp, is it supported now?

[woodx9]

Does any one have link to it? Appreciate your help!

[woodx9]

I want to use it on infill request for code.

[brandon-lockaby]

from ./main -h

  --prompt-cache FNAME  file to cache prompt state for faster startup (default: none)
  --prompt-cache-all    if specified, saves user input and generations to cache as well.
                        not supported with --interactive or other interactive options
  --prompt-cache-ro     if specified, uses the prompt cache but does not update it.

You just need to set --prompt-cache filename

llama-cpp-python, I noticed, enables the caching by default

[woodx9]

I don't need the prompt cache, The codes varies when people use it, is it supported now?