FR: Support OpenRouter (or any OpenAI compatible) Provider with model selector

[elianiva]

Currently it requires you to use OpenAI for its AI feature. It would be better if users can select which models they want to use.

I believe any OpenAI-compatible API should do it, OpenRouter is just one of the solution a lot of people use since they provide a lot of different models that you can choose.

Feel free to ask me more info if needed. I'm happy to help implementing this.

[jjmata]

Love this idea, as well as providing further LLM observability via Langfuse or even PostHog that recently introduced some. Also, while OpenRouter is probably the simplest and a drop-in replacement most likely ... one could also consider putting ollama in the compose.yml and opening the door to local models.

[hkamran80]

You could also add an environment variable to set the base URL for the OpenAI client and use any OpenAI-compatible API.

sure/app/models/provider/openai.rb

Line 10 in fb18521

@client = ::OpenAI::Client.new(access_token: access_token)

[elianiva]

You could also add an environment variable to set the base URL for the OpenAI client and use any OpenAI-compatible API.

sure/app/models/provider/openai.rb

Line 10 in fb18521
@client = ::OpenAI::Client.new(access_token: access_token)

I've thought about this before, but then we can't select which model to use 🙂
It's hardcoded to be gpt4.1, and different API uses different model ID (that would be openai/gpt-4.1 if it were openrouter instead of just gpt4.1)

[jjmata]

Putting on a future train for the time being. Could always change if we get clarity, but personally I'd rather get some LLM observability first (which might be a good time to refactor this!)

[jjmata]

Related: #116

[jjmata]

Related: #122

[jjmata]

See #126! 🚀

[Kryszn0]

"It's hardcoded to be gpt4.1"

Would it be possible to support integrating with a project like LiteLLM? Basically, LiteLLM can act as a "wrapper" for Ollama and turn locally hosted models in to "OpenAI friendly" endpoints.

So if Sure is hardcoded to use gpt4.1 then setting an LiteLLM config file like so

model_list:
  - model_name: gpt4.1
    litellm_params:
      model: ollama/llama3.2:1b             

Would effectively "transform" llama3.2:1b hosted on ollama into a model called/referenced as gpt4.1