How to pass a custom mask to my loss function?

[intercodesys]

I want to be able to use a mask in my loss function, so it focus on specific items more than others. I tried to define the mask and pass it to the loss function: loss_func = partial(my_loss, mask) which obviously does not work because the dataloader shuffles the items in every batch.

So my question is how do I do this? Can I pass the mask to the dataloader somehow and retrieve it from the loss function? Or can I get the shuffled indices of the current batch from within the loss function?

Thanks!

[vrodriguezf]

I'd have a look at how it is done in the implementation of the Masked Value Predictor callback.

…

On Fri, Aug 19, 2022, 21:59 intercodesys ***@***.***> wrote: I want to be able to use a mask in my loss function, so it focus on specific items more than others. I tried to define the mask and pass it to the loss function: loss_func = partial(my_loss, mask) which obviously does not work because the dataloader shuffles the items in every batch. So my question is how do I do this? Can I pass the mask to the dataloader somehow and retrieve it from the loss function? Or can I get the shuffled indices of the current batch from within the loss function? Thanks! — Reply to this email directly, view it on GitHub <#562>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABWBN2C5ZOFVMH2FNDDTWIDVZ7RRLANCNFSM57BZ3A5Q> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[intercodesys]

Thank you for your suggestion and I enjoyed to read the MVP code, but it seems like their mask always is a function of x, so in that case they do not need to know the indices of the current shuffled batch. In my case the mask is static, and is not a function of x, so it doesn't help. I tried to use mymask[self.dls.idxs] to get mask of the current batch, but it doesn't work if the trainsize % bs != 0 (last batch idxs always has same size as the bs)

[vrodriguezf]

what about enabling drop_last in the two dataloaders to skip the last batch?