How to generate embeddings for Documents using Text Data and also couple of meta data fields (categorical variables)

[chaituValKanO]

How to generate embeddings for Documents using Text Data and also a couple of metadata fields (categorical variables)? I want to later use these embeddings in MostSimilarDocumentsPipeline to retrieve duplicate documents in the document store.

I have checked TableTextRetriever. From the description, it looks like it can solve the problem. But unsure.
Any thoughts in this regard will be helpful

[JasperLS]

Once you have set up your document_store, you can add embeddings via update_embeddings like in this tutorial. Similarly, you may update meta tags with update_document_meta .

[chaituValKanO]

Can meta fields (categorical variables) be used for generating embedding along with Text data?

[JasperLS]

Yes, you can tell the retriever to also embed meta fields (filename for example, or any other arbitrary meta tag): https://github.com/deepset-ai/haystack/blob/main/haystack/nodes/retriever/dense.py#L1458