Quantizing Enhancement idea

[Swight1423]

This is a mini side project of my probabilistic compression experiments but this one is only attempting to improving accuracy of other lossy compression like quantization not compressing the data it's self. This is untested alpha software proceed accordingly(it does at least compile and had Gemini 12B look for issues but found nothing major). This targets FP16 sources, FP32 would take a lot of memory which might kill it's benefit. I am just posting this as a toy for someone to play with to see if it is beneficial or not.

The idea here is that you run ConstructInitialDictionary on the FP16 of the data you are going to quantize creating a frequency distribution. this can be done in chunks building up a dictionary representing all of them. you then pass this dictionary to ConstructConversionArray along with a distance penalty and it will produce an array of half values. when you decode a quantized file you can bit convert the results to Uint16 and use them as an index in the array from ConstructConversionArray to get the most probable neighbor based on your distance penalty and the frequency data.
The programming language used was VB.net because I am most familiar with it. Source code here:
(https://github.com/Swight1423/QuantizePlus)