De-quantize model

[fakerybakery]

Hi,
Is there any way to convert a q4_1 model to an unquantized model?

[KerfuffleV2]

It depends on what you mean. Quantization is lossy, the information that gets thrown away is gone forever. So you can convert your q4_1 model to float16 or float32 - the same format as the original model it was probably quantized from - but it'll still have the quality of a q4_1 model. Then if you try to quantize it again, or fine tune against it, etc, you'll be starting with the lower quality model.

If you still really want to convert it for whatever reason you can actually use the quantize tool. For example, converting a quantized model to f16 format:

./quantize --allow-requantize your_quantized_model.gguf fake_unquantized_model.gguf f16