Score function question

[peterandluc]

Hi,

1. The paper you amended https://arxiv.org/abs/1906.00363, seems not yet changed.
2. For the line 60th, why you minus 3? From the comment line 22rd, I assume that there three extra tokens, [CLS] and [SEP]*2. And I'm also curious that why you use math.pow instead of divided? What's the strength?
3. Can you please describe more about Inspirational Sentences.zip? Are you using this corpus for pre-training?

You have done a good job in this area. Thank you in advance.

[wangcunxiang]

Hi,

1. The paper you amended https://arxiv.org/abs/1906.00363, seems not yet changed.
2. For the line 60th, why you minus 3? From the comment line 22rd, I assume that there three extra tokens, [CLS] and [SEP]*2. And I'm also curious that why you use math.pow instead of divided? What's the strength?
3. Can you please describe more about Inspirational Sentences.zip? Are you using this corpus for pre-training?

You have done a good job in this area. Thank you in advance.

Sorry to see it so late.

1. I just checked it, it seems changed.
2. Yes, one [CLS] and two [SEP]s.; There is no mathematical benefit, it is only because we could change to other powers if needed when coding.
3. Yes, we used it to pre-train.
   Thanks for your comments.