What is the purpose of weight quantization and activation quantization?

[CoinCheung]

Hi,

I did not find enough specification of weight/activation quantization. I can thought of two usage of this: the first one is to speed up training with 8 bit or 4 bit computation, in which process we need to quantize the weights or activation. The second usage is the so-called quantized-aware-training(QAT), in which we want the model to be adapted to the precision of quantization during training process. Would you tell me which one is the purpose of weight/activation quantization in deepspeed?