Training a Dense Retriever Model: Embedding Retriever (sentence_transformers)

[nachoperezzv]

Hello there!

Anyone knows how to train a embedding retriever model? I've tried to train a sentence-transformers model with the dpr-squad data format, but I got this error:

ValueError: Some training examples don't contain the fields {'question', 'pos_doc'} which are necessary when using the 'mnrl' loss.

MODEL_FORMAT = 'sentence_transformers'
EMBEDDING_MODEL = 'sentence-transformers/multi-qa-mpnet-base-dot-v1'

If anyone could tell me which format the json needs to be in order to train this model correctly.

Thanks a lot!

[bilgeyucel]

Hey @nachoperezzv, the training data is different than SQuAD format. Instead, it should have:

question: the question string
pos_doc: the positive document string
neg_doc: the negative document string
score: the score margin

As the error message states, {'question', 'pos_doc'} fields are necessary for 'mnrl' loss. You can see the API Reference here for details.

[nachoperezzv]

Great! Thanks for the reference. Is there any method to transform from SQuAD format to training data? For example. There's squad_to_dpr to transform from SQuAD to DPR training format. Is there something similar for the embedding retriever format?

[bilgeyucel]

We don't have a similar functionality now, but feel free to open an issue (maybe even a proposal PR, you can check how proposals work here 👀 ), if you think that would be useful! 😊

[nachoperezzv]

I'll try my best. Just to be sure, these are the keys of the training data right?

• question: the question string
• pos_doc: the positive document string
• neg_doc: the negative document string
• score: the score margin

I guess that the pos_doc is what would be the positive_ctxs in dpr training format ? Would that mean that the neg_doc would be the hard_negative_ctxs ? Also don't know what the score_margin exactly means.

Also, each key only can have one value or could it be a list of values ?

{ "question": "XXXX?", "pos_doc": [ "XXX1", "XXX2" ], "neg_doc": "XXX1", "score_margin": [0,1] }

[bilgeyucel]

Hey @nachoperezzv, depending on the loss function, required fields change. You only need question and pos_doc for mnrl loss, but additionally, if you provide neg_doc, this information will also be used. You can check all loss functions and how they work here: https://www.sbert.net/docs/package_reference/losses.html#multiplenegativesrankingloss

The training works without problem when pos_doc is a list of values, but I feel this will not work properly behind the scenes. I'll let you know when I am sure.

btw, I see that you already marked an answer for the first question you asked but do you also have answer to your last question? Feel free to add here if you have any updates 🙂