hello, why the convolution part in the resnet is implemented in linear module?

[yoksen]

Question is as above, I dont know why the conv can be replaced by linear module? Could you please help answer this for a new learner in uncertainty?

[adityagoel4512]

Thanks for the question @xiexuchen. If you check out https://arxiv.org/pdf/2006.10108.pdf I believe the important point for distance preservation via spectral normalisation of hidden weights is to have a skip connection, so whether convolutional layer or standard matrix product is not quite as important.

[yoksen]

Thanks for the question @xiexuchen. If you check out https://arxiv.org/pdf/2006.10108.pdf I believe the important point for distance preservation via spectral normalisation of hidden weights is to have a skip connection, so whether convolutional layer or standard matrix product is not quite as important.

Yes, I understand what you mentioned, and now the problem I have is this: for some specific backbones, such as the wide-resnet28-10 mentioned in the paper, I would like to know how to change the backbone file to not only be able to set the spectral norm, but also to be able to set the upper bound of the spectral norm, so as to reproduce the results mentioned in the paper, can you provide some help? Thank you in advance.

Yes, I understand what you mentioned, and now the problem I have is this: for some specific backbones, such as the wide-resnet28-10 mentioned in the paper, I would like to know how to change the backbone file to not only be able to set the spectral norm, but also to be able to set the upper bound of the spectral norm, so as to reproduce the results mentioned in the paper, can you provide some help? Thank you in advance.

[yoksen]

Thanks for the question @xiexuchen. If you check out https://arxiv.org/pdf/2006.10108.pdf I believe the important point for distance preservation via spectral normalisation of hidden weights is to have a skip connection, so whether convolutional layer or standard matrix product is not quite as important.

I also implemented 'wide resnet with spectral normalization' according to my understanding, and the code is here: https://github.com/xiexuchen/Wide_Resnet_SN. I'm not sure if I have implemented it correctly or not, so I would like to ask for your advice. If you have time, could you please take a look and provide some guidance? The main modifications on the original wide-resnet28-10 are: (1) adding a spectral norm module outside each convolutional kernel, and (2) modifying the 'compute_weight' function in the PyTorch official 'spectral norm' file. I hope this hasn't caused you too much inconvenience and thank you very much.