Chat templates and llama-server #9741
Unanswered
tesseract241
asked this question in
Q&A
Replies: 1 comment
-
It's not separate, there's just one prompt for everything. Put the system one first, then append user prompt, then append the conversation. There's some prompt information in gguf files and it is shown in console output on startup but don't know if it can be used automatically. I just manually copied it. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Hello all, I'm trying to wrap my mind around how to use llama-server.
The chat template wiki page says
I'm not sure how to read this: are chat templates embedded into a gguf file used if present?
Let's assume the model has no chat template and I want to provide my own, should I still separate the system prompt (wrapped in its special tokens) from the user prompt (also wrapped in its special tokens and ending with the token that starts the AI reply) or just put everything in the user prompt instead?
Beta Was this translation helpful? Give feedback.
All reactions