How to initialize the ViT model with the pre-trained weights from CLIP

[Harry-zzh]

My current approach involves manually aligning the parameter names between ViT and the visual part of the CLIP model one by one. However, I wonder if there is a more streamlined method available within MMSegmentation. Thank you for your assistance.

[mm-assistant[bot]]

We recommend using English or English & Chinese for issues so that we could have broader discussion.

[xiexinch]

Hi @Harry-zzh,
Sorry, we do not provide such a tool.