2D Matryoshka Sentence Embeddings

[vasilisvyth]

Thanks for your great work. Could you please provide the code for 2D Matryoshka Sentence Embeddings or any checkpoints?

[SeanLee97]

Hi @vasilisvyth, we are still working on improving it. The code for the improved version will be released soon.

For checkpoints, you can try bellows:

• nli: https://huggingface.co/SeanLee97/2dmse-bert-base-nli-v1
• general: https://huggingface.co/mixedbread-ai/mxbai-embed-2d-large-v1

[dribnet]

Thanks - the 2dmse-bert-base-nli model loads and perform well for me using the full hidden size, which appears to be 768.

Can you confirm that the other usable hidden sizes on this model would be n={8, 16, 32, 64, 128, 256, 512} taken by clipping the full vector from v[0:n]?

[SeanLee97]

Update:

Hi @dribnet @vasilisvyth, I updated angle_emb to version 0.4.0 today. It's now much easier to use for inferring multiple types of sentence embeddings.

For Espresso (previously 2DMSE) or Matryoshka embeddings, you can find usage here: https://angle.readthedocs.io/en/latest/notes/quickstart.html#infer-espresso-matryoshka-models.

---

For @dribnet 's question:

Can you confirm that the other usable hidden sizes on this model would be n={8, 16, 32, 64, 128, 256, 512} taken by clipping the full vector from v[0:n]?

Answer:
You can use any embedding size for Espresso or Matryoshka embeddings as long as it is below the maximum embedding size.