What is better 13B q4_0 or 7B q8_0 ??? #1468
skidd-level-100
started this conversation in
General
Replies: 1 comment
-
In terms of perplexity q4_0 is always better than the 16bit version of the lower model. There are some examples in the readme and there's a discussion with comparisons for each model. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
I would like to know what kind of model is best for its parameter size,
based on file size I would guess that the 13B one is better.
please let me know
Beta Was this translation helpful? Give feedback.
All reactions