LiteLLM Compatible Adapter

[iamthebot]

We're using litellm to proxy various LLMs in OpenAI format. While there's already an OpenAI compatible adapter (which works just fine), a dedicated LiteLLM compatible adapter would enable prompt caching which is a big deal for performance (and cost saving).

[olimorris]

Ref: #800

[iamthebot]

Just to be concrete here, "If an adapter can leverage the openai_compatible adapter, then I will not be accepting a PR for it. Instead, I would ask the community to share it in the discussions and then we can link to it in the documentation."

Is your stance that a litellm-compatible adapter would not be accepted as a PR? This is a bit different than the typical LLM-specific adapter (where your stance seems quite reasonable) in that litellm can proxy a large number of LLMs. It has specific additional arguments + exposes the ability to control LLM-specific behavior that is not part of the OpenAI API standard.

[olimorris]

My primary motive for not including adapters is to ensure I don't get caught up in support hell where I spend my time just fixing adapter issues. Also, the more adapters I support, the longer it takes to add new features as was the case in #1141.

So unless there's a compelling reason as to why something should be included in the plugin, my default stance will always be "put it in the show and tell section in the discussions". It achieves the same outcome as I make a point to link to them in the docs pages.