How to quantize the KV cache ?

[Abhranta]

I want to quantize the KV cache and use it for Q4_0_4_$ type of moedls because these quant types don't need dequant and it might make it faster if the kv cache is also quantized to 4 bit. I cannot figure out how to quantize the kv cache though!!!!!

Any help ?