Fine Tuning vs. Feature Extraction

[gauravreddy08]

Hey Daniel, hope your doing great and staying safe !! I had a little doubt reagrding Fine Tuning vs. Feature Extraction

As you said in Fine tuning you keep a few (or all) the layers of the model unfrozen

But whats actually point of this ?

What does setting trainable=False do ?

Yeah after fine tuning we got a little bit inclination in the accuracy.
But is it necessary to fine tune your model to gain that small increase in the score ?

Or are there any other advantages in fine tuning. If so, what ?

[mrdbourke]

Hey Gaurav,

• trainable = False sets all of the layers in a downloaded/pretrained model to frozen, so these won't update any of their internal patterns when you start training on your data.
• For fine-tuning & keeping a few layers frozen you can use pretrained weights for your own dataset or, if your dataset is large enough, you can unfreeze the base layers (the already pretrained ones) and fine-tune them (tweak them) so they work better with your custom data - though many times feature extraction models (only top layers unfrozen) work quite well.
• Fine-tuning is not always necessary, it's usually used as a step to improve a pretrained model for your custom data, but it may not always result in large improvements.