custom quantization

[tusiqi1]

Can I use this project to customize the quantification of each layer of llama?

[cmp-nct]

Sure, just modify the quantize example, you can change quantization for layers or individual tensors.