Responses API and reasoning efforts support

[ArtyomShalagin]

Describe the issue

Hi!

My understanding is that mini-swe-agent currently always uses completions api. For agentic tasks OpenAI's reasoning models work much better with responses api. Also as far as I can see there is no support for reasoning efforts settings

I think both can be implemented with liteLLM's responses api: it can bridge to completions if provider doesn't support responses, and it can drop parameters (like reasoning efforts) if provider doesn't support a specific parameter. liteLLM's responses endpoint is in beta though, I'm not sure how stable it is

Are there any plans to support this? In my experience GPT-5 with responses api and high reasoning is very noticeably better than GPT-5 with completions api and medium reasoning so I'm very curious about benchmark results :)

[klieret]

Hi sorry for my late reply on this one. Interesting. One way we could do that is simply add a second model class for litellm with the responses API. Or if it doesn't work with litellm we could just directly add e.g., the openai API in here. I already wanted to add the --model-class argument to mini.py (you can already use it for swebench.py and others).

[ai-jz]

@ArtyomShalagin You can have a try of the Response API via #506. I shared performance comparisons between Responses API and Chat API. Surprisingly, Responses API doesn't lead to better performance in case of GPT-5 and actually hurts performance in case of GPT-5-mini.

However, we do observe that high reasoning effort boosts performance a lot: GPT-5-mini with high reasoning is nearly on par with GPT-5 with medium reasoning.

Btw, you can test GPT-5 with high reasoning effort without Responses API via the following setting:

model_name: "gpt-5-mini"
model_kwargs:
  drop_params: true
  reasoning_effort: "high"
  verbosity: "medium"