Difference between GGMLFType #5471
-
For ggml, there are many types. Just like Q{i} means to {i} precision. class GGMLFType(IntEnum):
ALL_F32 = 0
MOSTLY_F16 = 1
MOSTLY_Q4_0 = 2
MOSTLY_Q4_1 = 3
MOSTLY_Q4_1_SOME_F16 = 4
MOSTLY_Q8_0 = 7
MOSTLY_Q5_0 = 8
MOSTLY_Q5_1 = 9
MOSTLY_Q2_K = 10
MOSTLY_Q3_K_S = 11
MOSTLY_Q3_K_M = 12
MOSTLY_Q3_K_L = 13
MOSTLY_Q4_K_S = 14
MOSTLY_Q4_K_M = 15
MOSTLY_Q5_K_S = 16
MOSTLY_Q5_K_M = 17
MOSTLY_Q6_K = 18 |
Beta Was this translation helpful? Give feedback.
Answered by
fgdfgfthgr-fox
Feb 13, 2024
Replies: 1 comment
-
Check out #1684 |
Beta Was this translation helpful? Give feedback.
0 replies
Answer selected by
Starlento
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Check out #1684