How do I insert a <|START_OF_TURN_TOKEN|> in interactive mode?

[araleza]

I'm trying to use the new Command-R +, following the template:

<BOS_TOKEN><|START_OF_TURN_TOKEN|><|USER_TOKEN|>{prompt}<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>{response}

I can't seem to just type this as text and have it work, because I don't think (for example) <|START_OF_TURN_TOKEN|> is being turned into a single token, but instead is becoming a string of normal text tokens.

Am I missing a way to do this, or is it not possible in llama.cpp interactive mode?

[Jeximo]

is becoming a string of normal text tokens

Yes, you're not missing anything - it's fundamental to how main works.
There is a way to add the template for server within lama_chat_apply_template, here's the detail: https://github.com/ggerganov/llama.cpp/wiki/Templates-supported-by-llama_chat_apply_template#how-to-add-a-new-template

For main, llama_chat_apply_template is not implemented yet: #6391 (comment)

[araleza]

Thanks! And I appreciate the links too, @Jeximo.