Please add options for temperature in the settings!

[innovative-illusions]

It would be really nice to have a temperature option in the settings. Some of the default top-p values for models on the Hyperbolic api are incompatible with a temp of 0, so I cant use any qwen models unfortunately.

[BradKML]

If it is Qwen, better to go to Ollama to pick the right settings + promptset cus those could make RooCode more stable #441 (comment)

[BradKML]

@dosu could you figure out why using Ollama for self-hosted Qwen/DeepSeek can lead to failures in tool use, compared to OpenRouter as a 3rd party provider?
#314 #491 #533 #1508 #758

[dosubot[bot]]

Using Ollama for self-hosted Qwen/DeepSeek can lead to failures in tool use compared to OpenRouter due to several key factors:

1. Context Size Limitations: Ollama’s default context window is only 2048 tokens, which is too small for most Roo Code workflows. This can cause abnormal tool behavior or failures, especially for complex tasks. To avoid this, you need to explicitly set a much larger context size (at least 12k, ideally 32k) using the num_ctx parameter when configuring your model in Ollama. If you use the OpenAI-compatible API, you’re limited to 2k context, but Ollama’s native API allows you to increase this limit. OpenRouter, by contrast, handles larger contexts more gracefully and even offers a “middle-out” message transform to compress prompts that exceed the model’s context size, reducing the risk of failures in long or complex sessions [source] [source].

2. Temperature and Model Parameter Handling: Ollama sometimes defaults to a temperature of 0, which can make outputs less creative or even incompatible with certain models (e.g., DeepSeek R1, which may not accept temp=0). Roo Code now supports per-model temperature settings, but you must ensure these are set correctly in your configuration profiles. OpenRouter generally manages these parameters more flexibly and consistently, reducing the risk of issues caused by incompatible defaults [source].

3. Resource Requirements and Setup Complexity: Running large models locally with Ollama is resource-intensive and requires a powerful machine. Misconfiguration or insufficient hardware can lead to slow responses or outright failures. OpenRouter, as a cloud provider, abstracts away these hardware and setup concerns, leading to more reliable tool use for most users [source].

4. Model Capabilities and Features: Some features, like multimodal support or advanced prompt handling, may not be available or as robust in self-hosted Ollama setups. For example, DeepSeek is not a vision model and cannot interpret screenshots, which can limit its effectiveness for certain tasks compared to models available via OpenRouter [source].

5. Recent Fixes and Improvements: Roo Code has made improvements to DeepSeek R1 support and context window size handling, but you must be on a recent version and ensure your Ollama models are configured with the correct parameters for best results [source].

In summary, failures with Ollama are most often due to context size misconfiguration, temperature parameter mismatches, and the inherent complexity of running large models locally. OpenRouter avoids many of these pitfalls by managing resources and parameters in the cloud, offering a more seamless experience for most users. For best results with Ollama, ensure you increase the context size, set appropriate temperature values, and use up-to-date versions of both Roo Code and your models.

To reply, just mention my name @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}   

[akd-io]

I just learned about the problems with temperature 0 about a month ago. I assume this isn't getting enough attention, as other people just aren't aware, like me.

In my mind, temp 0 made sense as the preferred setting. Why would I introduce less-likely tokens into my generated code? As it turns out temp 0 can lead to "text degeneration", especially repetition. Increasing temperature as a solution didn't make sense to me, as that would make it possible for the model to pick suboptimal tokens when generating syntax, where one specific token is the only valid solution.

The problem is solved with top-p sampling (aka. nucleus sampling), where sampling will ignore tokens that are simply too unlikely, so you can get diverse generation during thinking and text writing, but precise generation during syntax/JSON generation and tool calling.

I believe temperature and top-p control could meaningfully decrease repetition and thereby increase quality in the text and code generated by Roo Code. I've personally found thinking models to be very repetitive, to the point of unusable, in Roo. They might be in and of themselves, it's impossible to make good generalisations about this stuff from personal experience, but temp 0 sure isn't helping.

It's very unfortunate that this setting isn't available, as it bottlenecks every single model, and it seems like nobody knows. I'll be using Aider till temp and top-p is in the settings.

I'll leave the abstract of the 2019 The Curious Case of Neural Text Degeneration paper here:

Despite considerable advancements with deep neural language models, the enigma of neural text degeneration persists when these models are tested as text generators. The counter-intuitive empirical observation is that even though the use of likelihood as training objective leads to high quality models for a broad range of language understanding tasks, using likelihood as a decoding objective leads to text that is bland and strangely repetitive.
In this paper, we reveal surprising distributional differences between human text and machine text. In addition, we find that decoding strategies alone can dramatically effect the quality of machine text, even when generated from exactly the same neural language model. Our findings motivate Nucleus Sampling, a simple but effective method to draw the best out of neural generation. By sampling text from the dynamic nucleus of the probability distribution, which allows for diversity while effectively truncating the less reliable tail of the distribution, the resulting text better demonstrates the quality of human text, yielding enhanced diversity without sacrificing fluency and coherence.

bold84 made a great PR, #1773, which I think was ignored by the Roo team as it didn't follow the issue-first approach. This should be a great inspiration for anyone writing a detailed feature request and/or implementation.

[BradKML]

@akd-io I think they have the ability to set the temperature now for RooCode but I am not sure about the other settings (Top-P vs Top-P vs Top-K vs penalties), and that everything from Ollama self-hosting to OpenRouter to LiteLLM-supported providers having different limitations to these settings

Side note on the settings https://ollama.com/mychen76/qwen3_cline_roocode

@dosu please confirm if this is the case, that RooCode updates have this issue covered

[akd-io]

Ah yes, I must have been a bit confused when writing my comments here. You are right about temperature.

The problem is top-p missing, as non-0 temp without top-p is pretty useless for code editing tasks.

[BradKML]

Extra finds for those thare are not "made" for RooCode/Cline, but more for general use with thinking models

• https://ollama.com/hhao/qwen2.5-coder-tools
• https://ollama.com/MFDoom/deepseek-r1-tool-calling
• https://ollama.com/ishumilin/deepseek-r1-coder-tools

[bold84]

@BradKML a bit misleading.
But likely unintentional, due to lack of knowledge.

I did, in fact, create cot_proxy primarily for Cline/RooCode. Initially, just to get rid of the think tags, as both didn't have support for them until later.

Later, I added the sampling parameter override feature as especially for Qwen3, thinking and non-thinking modes require different parameters.
This way you can transparently use both modes with the correct parameters without having to reload the model, even if the client doesn't support setting these parameters.

It solves the problem, regardless of which inference engine is used.

[BradKML]

@bold84 to correct myself, I meant that I have found other Ollama settings (treating Ollama as a provider here) designed to force-set parameters and prompts, such that RooCode/Cline dependency can be decoupled. For other non-self-host platforms, it likely requires config adjustments in the application, or proxies like what you have demonstrated.
I wonder tho if LiteLLM can be treated as a better option since that could reduce duplicated work. Client-circumvention is interesting as a key topic, considering how most laymen assume that there are no "parameters" to set, treating whatever comes out of the API as immutable.