Use default stringtemplate delimiters for prompt templates

[pherklotz]

The current set delimiters { and } do not follow the stringtemplate default and need escaping if you want to build a prompt with JSON (as described in issue #355). This pull request will address this issue.

[youngmoneee]

Upon reviewing the PR, I've considered that using { and } as delimiters in the current template writing approach might offer the advantage of allowing variables to be used directly in the frontend layer with frameworks like React and Vue, without the need for additional modifications.

Furthermore, since this delimiter usage is common across several frameworks, it could also lead to improved readability. It seems to me that this might have been the author's intention.

nit

[markpollack]

I don't follow the comment regarding { and } offering an advantage for frontend frameworks. Do you mean sending frontend code as part of the prompt? JSON is a more common case than embedding react code.

Between easier json embedding into the prompt or sending react code, I'd go with easier json embedding. That said, it sounds like the delimiter should be configurable to better accommodate all cases.

My intention was to support the python F-string syntax. How does python handle the case of inserting json into a f-string?

I'd need to reproduce the original issue, what does embedding json into the TextTemplate look like?

[youngmoneee]

What I intended was similar to what you mentioned about F-String, as various languages or frameworks use { , } based variable templates. This makes it useful when importing templates from external sources or exporting them, and given its widespread use, it should be easy for developers to read.

Changing the delimiter to <, > due to the difficulty of embedding JSON in prompts might introduce new challenges when inserting HTML or XML.

To fundamentally solve these issues, developing custom parsers, renderers, or using flexible variable templates through the injection of ST with various delimiters might be a good approach.

[pherklotz]

I agree, that it might be the best to introduce a configuration option for the delimiters.
I can change the PR accordingly. The default option would still be < and >. I agree that this introduces error with HTML or XML snippets. But I assume these cases are rarer than JSON snippets.

@markpollack an example prompt could be:

List all European countries and their capitals. Answer with the following JSON format: {"Country name": "capital name"}

This prompt fails if used with the ST Template Prompt mechanism.

[ThomasVitale]

I shared some thoughts on the issue for this PR about the templating strategy: #355 (comment)

[pherklotz]

So how do we proceed with this? I am fine with changing it, but will only start working once there is a commitment to a solution ;)

[youngmoneee]

So how do we proceed with this? I am fine with changing it, but will only start working once there is a commitment to a solution ;)

I have created a code that functions similarly to F-string.
It's a simple prototype, so it's not perfect beyond the basic functionality, but based on test code, if there's a specific scenario that needs to be addressed, it could be refined into a more advanced form.
If there is a situation that needs to be handled, it would be good to talk together. :)
#355

[HzjNeverStop]

I agree that PromptTemplate should provide an API for users to use different delimiters based on their own scenarios. However, can we keep the default delimiter as { } to avoid unnecessary compatibility changes for existing users?

[markpollack]

I also agree that we should allow different delimiters, and I've also had requests for different template engines. I do not want to change the default { and } as it would bring too much pain to users. If I made bad choice there, apologies. Bumping this to RC1 - sorry, it keeps slipping. Will get to it eventually! Apologies.

[coderphonui]

Voted for this. The { and } characters are used a lot to put examples for the JSON output of the prompt. Using them by default will cause a lot of pains for the user.

[markpollack]

Given the history and the default usage of { and } in the llm ecosystem, keeping the default to be the curly brace seems best to me. I will look into how to make this configurable, now that we have ChatClient it isn't immediately obvious where to enable such a feature but will look into it.

[markpollack]

Closing this PR but leaving the relevant issue #355 open