The precision issue of re-quantization.

[lingyezhixing]

For a 14B decode-only model, is there a difference in the final accuracy of the IQ3-M model when quantizing from f16 or bf16 to IQ3-M, and from IQ4-XS to IQ3-M? If so, what is the extent of the difference?

[wooooyeahhhh]

it depends on the precision the model was trained in so if it was done in bf16 then converting to fp16 then into something lower will result in a higher quality loss though it will be negligble so it might not matter