Question About the Calculation of code_lens in MeshTransformer

[qixuema]

Hello Phil,

I have been exploring the code in the repository and came across a line that piqued my curiosity regarding the calculation of code_lens. Specifically, the line is:

code_lens = ((codes != self.pad_id).cumsum(dim = -1) == 0).sum(dim = -1)

As I understand it, this line appears to calculate the number of non-padding elements in codes. However, I noticed that right after this, code_lens is used to set the eos_token_id.

codes[batch_arange, code_lens] = self.eos_token_id

Based on the above calculation method, it seems to add the eos_token_id at the very beginning of the codes sequence, rather than at the end. This seems a bit counterintuitive to me, as typically, eos_token_id is added at the end of a sequence.

Since I'm not very familiar with this part of the functionality, I'm curious about the rationale and logic behind this particular calculation method. Is there a specific reason or scenario for this approach, or am I misunderstanding the code?

Looking forward to your clarification, and thank you for your contributions to this project!

Best regards,
Xueqi

[lucidrains]

@qixuema hi Xueqi

yes, i do believe that is a bug 🐛 , thank you for catching it! the latest commit should be more sound