About the TFLOPS number in Hardware Settings

[KohakuBlueleaf]

cc @Vaibhavs10

It looks like the TFLOPS value in Hardware Setting is based on "TFLOPS of 'vector unit'" (such as GPU in M-series chip, or CUDA core in Nvidia cards). While lot of modern hardware/gpu have specific hardware designed for dealing with matrix multiplication (which cost most TFLOPS inside modern NN as well).

Is there any reason that we don't use the TFLOPS number of Matmul core (Tensor Core, Neural Engine, XMX, NPU...) but vector core?

I have suspected that we want to see the "FP32 TFLOPS" but looks like the baseline is actually FP16 since Tesla T4 have ~60TFLOPS.