Replies: 1 comment
-
This is obsolete, given the session interface |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Currently the inference API is defined by a named inference schemas "llama" being different from "whisper". In this particular case this difference makes sense, but the borders become fuzzier when we start talking about model classes as in "text-to-text LLM", "TTS model", etc.
So instead of differentiating at root level, I propose we define instance interfaces. So, a model would then support instance interfaces as opposed to a particular schema. Then a loader can be chosen by an instance interface, say "chat", or "tts".
This has a lot of implications on schema generation and the associated codegen. Much more specification is needed
Beta Was this translation helpful? Give feedback.
All reactions