Does Kong Manager not support model discovery? #416
Unanswered
sebastianrueckerai
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
I tried and failed to use Kong Manager as a Gateway in front of openwebui.
It seems that part of the issue is setting up model discovery.
As far as I can see the AI Proxy / AI Proxy Advanced plugins only support the chat and completion endpoints.
What about the use case were I want to give some users access to groqs models: I have a single api key. Kong Manager connects to groq trough this api key. Users connect to Kong trough openwebui and get all the models provided by groq. Is this possible?
Also, I found no tutorial about having Kong Manager play nice with openwebui. This seems like a big use case, maybe someone can provide a way to do this?
I am looking for a faster alternative to LiteLLM - which fills this use case exactly but is slow.
Beta Was this translation helpful? Give feedback.
All reactions