Openrouter or custom model with Openapi compatible

[yamijuan]

Is there any way or plan to add open router as model provider or be able to set up a custom model provider that's openapi compatible? thanks

[mrT23]

We support a lot of different providers:

Openai-compatible providers (like vllm) are supported, among others, via litellm
https://qodo-merge-docs.qodo.ai/usage-guide/changing_a_model/#ollama

[user753]

@yamijuan Did you figure out how to use openrouter?

[yamijuan]

I haven't implemented it yet, but I read the documentation, and seems clear what to do with litellm

[mrT23]

if open-router (which i have not used) are providing openai-compatible mechanism, that it is fully supported with pr-agent

something like:

[openai]
custom_llm_provider ="openai"
api_base ="..."

[config]
model="openai/Qwen2.5-32B"
model_turbo="openai/Qwen2.5-32B"
fallback_models=["openai/Qwen2.5-32B"]

[Ocrosoft]

I encountered the following issue when using the third-party OpenAI interface.

[config]
model = "openai/gpt-4-0613"

This is part of the configuration file, and then I receive the following error:

pr_agent.algo.utils:clip_tokens:924 - Failed to clip tokens: 'Could not automatically map openai/gpt-4-0613 to a tokeniser. Please use `tiktoken.get_encoding` to explicitly get the tokeniser you expect.'

Perhaps we need to remove the "openai/" part when calling encoding_for_model?

[yamijuan]

if open-router (which i have not used) are providing openai-compatible mechanism, that it is fully supported with pr-agent

something like:

[openai]
custom_llm_provider ="openai"
api_base ="..."

[config]
model="openai/Qwen2.5-32B"
model_turbo="openai/Qwen2.5-32B"
fallback_models=["openai/Qwen2.5-32B"]

I followed this, while not the way litellm adds openrouter, this worked

[yamijuan]

I've struggled with this a lot, reached a point where routing is being done to open router but litellm fails to send auth credentials

[irfansofyana]

anyone know what config looks like if we use openrouter?

[irfansofyana]

after some struggle, I can found the rootcause and create this PR: #1744 it should solved the auth error