What is the meaning of the hist when doing the quantize?

[Tr-buaa]

the terminal like this:

lama_model_loader: - kv 16: tokenizer.ggml.eos_token_id u32
llama_model_loader: - kv 17: tokenizer.ggml.padding_token_id u32
llama_model_loader: - type f32: 65 tensors
llama_model_loader: - type f16: 226 tensors
llama_model_quantize_internal: meta size = 1248800 bytes
[ 1/ 291] token_embd.weight - [ 4096, 55296, 1, 1], type = f16, quantizing to q4_0 .. size = 432.00 MB -> 121.50 MB | hist: 0.037 0.016 0.026 0.039 0.057 0.077 0.096 0.110 0.116 0.110 0.096 0.077 0.057 0.039 0.026 0.021
[ 2/ 291] blk.0.attn_q.weight - [ 4096, 4096, 1, 1], type = f16, quantizing to q4_0 .. size = 32.00 MB -> 9.00 MB | hist: 0.037 0.017 0.028 0.042 0.058 0.076 0.092 0.106 0.114 0.107 0.093 0.077 0.060 0.043 0.028 0.022
[ 3/ 291] blk.0.attn_k.weight - [ 4096, 4096, 1, 1], type = f16, quantizing to q4_0 .. size = 32.00 MB -> 9.00 MB | hist: 0.035 0.012 0.020 0.032 0.049 0.071 0.098 0.126 0.143 0.126 0.098 0.071 0.049 0.032 0.020 0.016
[ 4/ 291] blk.0.attn_v.weight - [ 4096, 4096, 1, 1], type = f16, quantizing to q4_0 ..

I have a question that what is the meaning about the hist? I can't understand it when I read the code. Can you help me？

[ggerganov]

For 4-bit quantization, there are 16 integer values (2^4: -8, -7, -6, ..., 5, 6, 7).
This histogram shows the distribution of the integers for each tensor when it is quantized.

[Tr-buaa]

oh, I understand it, thanks!

[rikoras]

Perhaps the distribution information of the integers can be used for some debugging or analytical purposes?

By the way, judging by your ID, I’m curious if you’re from the same institution as I am…

[Tr-buaa]

oh, thanks! And , what year are you in BUAA？

[rikoras]

First-year graduate

[Tr-buaa]

So do I, and also first-year work.