Best config to run on Apple hardware? #8728
Unanswered
pgorzelany
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Hi there! First of all, thank you for this project, you guys are doing some great work!
I am writing a MacOS and iOS app designed for Apple Silicon. How should I configure llama to have the best performance on Apple platforms? I am using the library as a Swift Package , specifically using the C api. What are the llama_model_params and llama_context_params or any other config that will give me the best performance? Or maybe the default config is good enough?
Thanks!
Beta Was this translation helpful? Give feedback.
All reactions