Does this tool work with quanitzed models? #482
bartowski1182
announced in
Q&A
Replies: 4 comments
-
|
That's exactly what I was looking into. Specifically GPTQ models like: https://huggingface.co/TheBloke/based-30B-GPTQ |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
I also wondering how to start the quantize work. Looking forward to some helps. |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
looking forward to update |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
Looking forward to support quantized models. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Haven't found any reference so I assume no, but wanted to check. would be really cool but also presumably insane levels of work
Beta Was this translation helpful? Give feedback.
All reactions