What kinf of prompt should I use when using llama-2-7b?

[vickychen928]

When using llama-2-7b-chat, I can use the following instruction to test my model:

./main -m models/llama-2-7b-chat/llama-2-7b-chat_q4.gguf --color -f prompts/alpaca.txt -ins -c 2048 --temp 0.2 -n 256 --repeat_penalty 1.3 -ngl 100

but when I change my model to llama-2-7b, I can't use the instruction above. If I use the above instruction, I will get a weird response consists of lots of blank response. I guess the problem may be prompts/alpaca.txt doesn't match llama-2-7b. So I'm wondering which prompt should I use to test llama-2-7b?
Thanks!

[Jeximo]

The chat version is diverse enough that it works even with an alpaca prompt template, but the base model does not have a prompt template - it's more of a text completion engine.

To use base model, then remove -ins, and -f prompts/alpaca.txt. Add -p "The sum of 5+5 is". To get the most from the chat version, then try the following:

Remove -ins, and -f prompts/alpaca.txt. Add -i -e --in-prefix "[INST] " --in-suffix "[/INST]" -p "[INST] <<SYS>>\nYou are an Assistant named Llama.\n<</SYS>>\n\n[INST] Hello Llama.[/INST]Hi, how can I help you?</s>"

[BBC-Esq]

Here's the prompt format I use and it seems to work fine:

prompt = f"<<SYS>>\n{system_message}\n<</SYS>>\n\n[INST] {user_message} [/INST]"

If I recall, I got it from the config information and/or the readme on huggingface...