a question about paper

[pengzha0]

hello , i have a question about that why you don't use normal convolution in the first layer

[kartikgupta-at-anu]

Can you elaborate what you mean by normal convolution layer?

[pengzha0]

thank you for your reply!
I have some new questions for your paper
1.In the article, you mentioned that your optimization goal is to make the singular values of the Jacobian matrix closer to 1, which leads to better signal propagation in the network. However, in your experimental results (as shown in the figure below or table13 in the paper), it can be seen that under the conditions of BNN-WAQ, the mean of the Jacobian matrix is already very close to 1. Therefore, I want to ask whether this indicates that BNN-WAQ itself already has good signal propagation capability.

2.The issue of poor signal propagation exists in both full-precision networks and binary networks because the probability of the correct class is extremely close to 1. However, why does only the binary network suffer from gradient vanishing while the full-precision network does not?

[kartikgupta-at-anu]

This is mainly since the JSV for BNNs is only an approximation since there is STE used for non-differentiable part of quantization function. Also, as mentioned in the paper, there are other things at play such as error signal.

[pengzha0]

thank you very much！I still have a doubt about error signal.
In BNN-WAQ, the error signal is close to 0 because the probability of the ground truth class is close to 1, but this problem also exists in full-precision networks. So Why does small error signal only affect binary networks but not full-precision networks?
hope to receive your reply！