ForBo7 // Salman Naqvi - Intuitively Approaching Einstein Summation Notation

[giscus[bot]]

ForBo7 // Salman Naqvi - Intuitively Approaching Einstein Summation Notation

An alternative way to write matrix operations.

https://forbo7.github.io/forblog/posts/16_einstein_summation_notation.html

[Max-We]

Nice write up, I like that you provided a new (intuitive) approach to explaining the Einstein Notation. Just a few things I found while reading, which you may correct:

• Under the paragraph Hadamard Product of A and B Transposed there seems to be a Latex type ("rightarray")
• In the paragraph Hadamard Product you write: "To access the element 8 in matrix A, we need to return the second row and first column.". I'd be more specific here because it's not the second row, but the row with the index 2 (the "third" row so to speak). The same goes for the column.

I like the in-depth discussion about rows and columns (thinking about axis vs iterators) and I'll definitely think about it in the coming days.

[ForBo7]

I appreciate the feedback! Good to know the guide is clear. Did you get confused or lost at any point?

I'll fix the Latex typo and tweak that paragraph to be more clear, too.

[Max-We]

At first, I was a little confused about your point of k denoting the rows instead of the columns in the first matrix of your example. In my understanding, it was actually denoting the columns. It wasn't really a problem to me in the first place because I already thought about k in ik,kj -> ij being dependent on i (the row index) in the first place, so while it's true that k is the column index and we'd like to multiply rows from A with columns with B, in this notation it actually means that rows of A are being multiplied and added up with the columns of B because as you pointed out later, in ik,kj -> ij, k is "dependent" on i (rows) in A and j is "dependent" on k (columns) in B, so overall we multiply rows of A with columns of B, and then sum it up.

[ForBo7]

Ah, right, I see. The reason I took k to be rows was because matrix multiplication involves taking the dot product of each row in the first matrix with each column in the second matrix.

If we use the following rule with matrix multiplication...

Repeating letters between input arrays means that values along those axes will be multiplied together.

...the axis that k represents must be the rows in the first matrix, and the columns in the second matrix, to stay consistent with matrix multiplication as defined above.

A matter of interpretation I suppose heh (I can see how k can denote the columns); but I'll rewrite this in the post to be more clear.