What model 'init' methods can we use in tsai and how?

[marcuswang6]

Hi, I was trying to tune my model about the weight_init methods, but don't know how to do that in tsai. Can I just assign init = 'torch.nn.init.kaiming_uniform_' ? And what weight init methods can be used here?

[zmce2018]

Hi @Waterkin,

I think initialization is the default setting in layers from pytorch if it is not stated otherwise.

For instance, if you can scroll down the code a little bit, you should be able to see the code:

# Model
        if init is True:
            init = nn.init.kaiming_normal_

This initialization will be passed down to the model by the utils.py in dir: "tsai/tsai/models/"
I believe you can make init to True and apply a proper initialization method.

Hope it helps.