[Feature]: Support reranker with vllm provider

[federicovallucci]

The Feature

Hi Everyone,
would like to know if it is possible to add in the stable version of litellm a provider for open source reranker (vllm/ or even openai/). I saw your documentation on how to add a new-reranker-provider, and it seems to work.

Motivation, pitch

I wanted to check if it is possible to add support for open-source reranker into main, so it is available for everyone. In that way, I won't need to rebase every time there are new updates and still want use open-source reranker.
Thanks

LiteLLM is hiring a founding backend engineer, are you interested in joining us and shipping to all our users?

No

Twitter / LinkedIn details

No response

[harvardfly]

@federicovallucci I've also found that the open-source rerank model deployed with vLLM cannot be used through LiteLLM. How are you addressing this issue now?

[federicovallucci]

hey @harvardfly ! I have cloned the litellm repo and added some custom code following this: add reranker provider. However, having the custom code is not ideal cause I need to rebase when there are new updates.

From a client side, I am able to send request using curl or requests python pacakge, but I am not able to do the proper modification to use litellm directly. Not ideal, but ok for my use case.

[i-makashev]

Here is mentioned that we can configure pass-through endpoint, but I can't figure out how to make it to work. I get different errors: Unsupported provider cohere_chat, invalid api_key and so on.

[RWDai]

i need this too

[andresC98]

@ishaan-jaff @NANDINI-star any updates if this is planned to be supported?

[subnet-dev]

Add Hosted VLLM rerank provider integration PR -> #9249

[flefevre]

we are also interessting in reranking model through Litellm to integrate it in Open WebUI https://docs.openwebui.com/getting-started/env-configuration/#rag_reranking_model

[NiuBlibing]

Is there any update for litellm server ui?

[kerlion]

I setup Qwen3-Rerank via vllm, can IiteLLM proxy it?