Implementing only CPU Usage #184
Closed
AnimeshDeb
started this conversation in
General
Replies: 1 comment
-
@AnimeshDeb Using the version 3 beta you can sepcify |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Hello, I am using the node js binding for a llama 2 chat model I found on the hugging face website, specifically from TheBloke. I have implemented a chat bot using the llama-2-7b-chat.Q5_K_S.gguf model from https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGUF . I want to ensure that only CPU is used when the AI model is running. Any advice on how to achieve this as I am relatively new to using AI models and optimizing their performance.
Beta Was this translation helpful? Give feedback.
All reactions