Home Assistant integration #575
Replies: 2 comments 11 replies
-
Yes, I would say an OpenAI compatible API, we haven't committed to an Ollama compatible API yet, but would be open to it (we run llama-server or vllm server under the covers).
We would certainly need the help here but yes I agree. |
Beta Was this translation helpful? Give feedback.
-
It's an AMD 7600 XT, and I'm running with
```
devices:
- "/dev/dri" - "/dev/kfd"
```
…On Thursday, January 16th, 2025 at 9:29 PM, Eric Curtin ***@***.***> wrote:
What's your GPU?
—
Reply to this email directly, [view it on GitHub](#575 (comment)), or [unsubscribe](https://github.com/notifications/unsubscribe-auth/AABJFINOMV72FY7LH6J6LO32LAJCTAVCNFSM6AAAAABU76PBKSVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTCOBVHEZDSNY).
You are receiving this because you authored the thread.Message ID: ***@***.***>
|
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
I got here from your shoutout on the ollama vulkan issue. I figured I'd continue here.
I'm working on a raspberry pi mini ITX board that supports discrete AMD GPUs to run LLMs inside home assistant. But there is no rocm for arm.
This needs several puzzle pieces:
So I'm very interested to explore what ramalama brings to the table here. From what I understand you're basically running llama.cpp in a container with an ollama compatible API, correct?
Do you support the native Ollama/OpenAI function calling features or are you indeed depending on the inference engine or client side templates to support this?
My current implementation is llama.cpp + home-llm which works but the function calling isn't great.
What I think could be potentially very interesting is a ramalama HA Integration/addon that can automatically launch the right inference engine.
Beta Was this translation helpful? Give feedback.
All reactions