How can I start a server that uses one gguf for the model, and a separate gguf for the embeddins target? #7721
-
|
running a server of a gguf has been pretty simple to setup. I know that embeddings are supported, but the --embeddings flag doesn't seem to accuept a separate gguf for embeddings specifically. If I want to use a llama3 model as the main model, but nomic-embed for the embeddings target, how is that accomplished? |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 1 reply
-
|
Currently this is not supported. You can try to start a second instance of |
Beta Was this translation helpful? Give feedback.
-
|
Thank you, that's what I expected but I couldn't find it explicitly stated. |
Beta Was this translation helpful? Give feedback.
Currently this is not supported. You can try to start a second instance of
server