More customizable/customized System Prompts for different models

[cstroliadavis]

So, I've been finding that some LLMs respond better to some prompt practices than others. Having a one-size fits all approach to the system prompt might be less useful as a result, costing more money than necessary and reducing the quality of results with some LLMs over others (I find that the current system prompt is ideal for Sonnet, but some of the others have a harder time with it, unfortunately, sonnet is one of the pricier models).

It would be nice if there were options for us to be able to have more refined control over the system prompt. Perhaps there could be an advanced option that allows us to completely replace the system prompt if we choose to do so. This could be saved within a configuration profile, or perhaps we could actually link some of these with specific models if we desire (bonus).

It would be good if this prompt had the capability of still working with the other prompt parts such as role, custom instructions, mode specific instructions, etc. Perhaps by adding tokens like {{roleDefinition}}, {{customInstructions}}, etc into the custom system prompt.

Something like this might also allow devs who might have other issues with the system prompt (see #558) to more easily make those modifications themselves until such time that code updates can be made.

[omicronns]

This. I was also thinking about some preprocessing step of the system prompt as @cstroliadavis proposed. Problems I faced:

• It is confusing how the prompt will look like at the end, currently you basically have no idea what will happen unless you diff two fully evaluated prompts. Such proposed "tag" system, will disambiguate it. It will give user possibility to see, what will be added, where and rearrange it. It will also allow to make overrides global, as far as I know currently when you override system prompt it is only possible on per workspace basis and basically fixed (system info, cwd).
• Despite overriding system prompt, you have no control over what Roo is telling the model, during back and forths when for example it didn't use any tool when it was expected by Roo. I'm not sure how to tackle this one.
• Enabling/disabling tools is ambiguous. Example: I create custom mode, and override system prompt for it. I want this mode to not have ability to write to files, so I remove tool description from prompt file. But then if there is no "edit" tool declared in the mode config json, note is being appended to environment_details: NOTE: You are currently in 'Simple' mode, which does not allow write operations. To write files, the user will need to switch to a mode that supports file writing, such as 'Code' mode.. But for some reason despite not having other tools in this mode also, they are not being noted as not available. Very confusing. Such information could be directly written by mode creator into overridden system prompt, I'm not sure why such behavior is hardcoded into Roo. Now the solution would be to enable all the tools, to have full control over initial prompt which seems "hacky". Also I'm not sure if it will fully mitigate the issue (see previous bullet).

I think, that treating default modes as special is part of the confusion. They should be pulled out of codebase and treated equally with custom ones.

Sorry if I said something that isn't true, I just started playing with Roo. But since beginning, system prompt handling was extremely confusing to me.

Also related issues: #2798 #1482 #2388

It seems, some work was already done regarding replaceable tags: #2366

[cstroliadavis]

Hey @omicronns , one thing that has been implemented that can help is the "Advanced: Override System Prompt" (twisty right under preview system prompt) or create a file .roo/system-prompt-name where name is the name of the prompt (e.g. code, ask, etc)

This will completely override the system prompt (minus things given to the AI like file lists and such)

What I tend to do, is create a prompt that is close to what I want, then I copy and paste the whole "system prompt" from that prompt and paste it into a file with that name. Then I begin tailoring as needed for a particular model (I may create a few different prompts tailored for different models I use and switch as needed).

It's a little bit of effort, but it's usually helpful.

[omicronns]

Thanks for response. I'm totally aware of ability to override system prompt in this way. This is pretty much what I was doing. But the process could be easier and clearer for newcomers. Now you have to spend some time reverse engineering how actually prompt is being generated, if you want actual control over what is happening and yet still surprises await like the NOTE I mentioned. Another example: one would expect when you view system prompt preview and copy it to override file, it would be exactly as when not overridden, but in the preview besides "core" prompt there is also "Role Definition" included there, so if you just copy that it will be duplicated.