I want to fine-tune open source models to learn how to use roo

[openSourcerer9000]

Every discussion of coding agents, I hear that Sonnet is the only thing that really works - not because it's necessarily smarter but just that it actually performs tool calls correctly. I've been trying roo out with a variety of local models which produce great code and take smart debugging steps, but they inevitably get hung up on tool calling. apply diff inevitably fails, and I have yet to see it actually query the internet like it's supposed to. The other issue is that context window tends to balloon, it seems the system prompts alone account for ~17K tokens! I believe both of these issues could be solved by simply fine-tuning these models to use roo. Below is my thought process on accomplishing this, if some direction seems off to anyone with more experience on this, please let me know!

I would want to start with the new devstral, since it seems to work best out of the box right now, as it was actually fine-tuned to use openhands agent tool.

This documentation seems to be pretty universal for LLM adapter datasets? I would likely be implementing qLoRA training in MLX (the weights should in turn work with any backend). https://swift.readthedocs.io/en/latest/Customization/Custom-dataset.html

I can create some synthetic data but it would be really hard to avoid hallucinations in the data, which is the whole issue in the first place. The gold standard would be if we could gather logs of Claude going through successful coding/debugging sessions, and calling tools correctly. I don't use Claude though. Does anyone already have logs like this they would be willing to share, or would be interested in recording some to collaborate on this project? This would also be an invaluable reference to upsample with synthetic data. I will be happy to open source the resulting model adapters - we need local options once BigAI's investors decide it's time to cash in on their investments!

EDIT

This is how you export a log of messages within the task to a .md file.

If you'd like to contribute, upload logs of any successful tasks using Claude or any other model to this repo: https://github.com/openSourcerer9000/RooCodeLogs
The repo is public FYI, if you don't want your logs to be public, but do you want them included in the models training data, shoot me an email at openSourcerer9000@gmail.com with them attached.

[dosubot[bot]]

Roo Code exposes logs through the VS Code Output panel (View → Output, select "Roo Code") for troubleshooting during usage, and if you're using the Requesty platform, you can access your coding stats dashboard and LLM interaction logs there as well. For models run via LM Studio, Roo Code tracks token usage and you can check LM Studio developer logs for more details if things go wrong [source] [source] [source].

For fine-tuning, Roo Code doesn't have built-in support for model training, but you can create custom modes and prompts to shape agent behavior. Use the Prompts tab or manually edit the custom_modes.yaml/.roomodes files to define tool access, role, and instructions—this can help you generate more realistic training data or synthetic logs for your adapter datasets [source] [source].

To mitigate hallucinations in synthetic data, minimize manual data dumps, break large tasks into smaller sessions to manage context window size, and validate tool outputs before including them in your dataset [source] [source].

The structure of Roo's prompts and tool calls is compatible with LLM adapter datasets: system prompts define capabilities and tool schemas, user messages provide context, assistant messages include tool calls, and tool messages return results. You can customize system prompts for each mode by placing a .roo/system-prompt-<mode_slug> file in your workspace [source].

I couldn't find any public logs of successful Claude coding/debugging sessions, but the Roo Code Discord and subreddit are active places where you might find collaborators or people willing to share logs or record sessions [source].

There isn't Roo-specific documentation on qLoRA training in MLX, but your plan to use the universal LLM adapter dataset format and qLoRA in MLX is sound. If you need more detailed MLX/qLoRA guidance, you may want to check the MLX and qLoRA communities directly.

Let me know if you want more details on any of these steps or examples of Roo's prompt/tool message formats!

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}   

[nikhil-swamix]

This will get a major Optimization, as they are trying to natively integrate structured outputs...
just search for related issue... you can see the discussion happend there.
ollama does have adapters when creating modelfiles. so ollama can be used.
most of the system prompt is just XML tools usage instructions, which i believe comes from original cline days, and has not changed much!