[Question] Does the training token count include image tokens and masked tokens? #136
Description
According to the DeepSeek-VL2 paper, approximately 800B tokens were used for training. I'm wondering if this includes image tokens and masked tokens, or if it only counts tokens where loss actually flows through.
Does anyone know about this? Thanks.