Q: Supported quantizations for MacOS Metal ? #2320
Answered
by
ggerganov
ianscrivener
asked this question in
Q&A
-
|
Previsouly only q4_0 and FP16 were supported... is that still the case? |
Beta Was this translation helpful? Give feedback.
Answered by
ggerganov
Jul 22, 2023
Replies: 2 comments 1 reply
-
|
The supported quantizations by Metal are listed in the following |
Beta Was this translation helpful? Give feedback.
1 reply
-
|
thanks @ggerganov |
Beta Was this translation helpful? Give feedback.
Answer selected by
ianscrivener
-
|
For the lazy folks: |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The supported quantizations by Metal are listed in the following
switch:https://github.com/ggerganov/llama.cpp/blob/5d500e8ccf5eee3de3ae66685cc3be75e43e08b9/ggml-metal.m#L646-L724