Positional Encoding

[schonIs]

I am currently conducting research on applying Transformer models to EEG signals. In your paper, I learned that by combining CNN and Transformer, it is possible to learn both global and local features. However, my question is why positional encoding is not added to the data input into the Transformer after the convolution with CNN. Could you please explain the reason for this?

[eeyhsong]

Hello @schonIs,

Happy to see you find this interesting part. I've also tried some position encoding techniques, but there was no significant improvement. I think that's due to the convolution layers performing a role for capturing position information.

Best regards,
Song

[schonIs]

Thank you for your quick reply and opinion about my question.