Faster causal matrix multiplication

[sh0416]

Causal matrix multiplication is widely used in transformer decoder.

It computes X @ Y, and masked out the upper triangular values.

This implementation computes the value that are masked out.

Or, is there any approaches to avoid that computations?