Using elasticsearch in offline mode when training and deploying DPR ? -Urgent please

[shahad2099]

Hello everyone ,
hope you're doing well!
I have question :
how can i used the indexed documents using elasticsearch in offline mode , to be more clear I want to index the docs and when the retriever need to get the answer search without connecting to elasticsearch . because i facing problem to connect to elastic-search locally so i use google colab to index the data but when I want to train DPR and create Q-A pipeline google colab stop working !?
if there is no way to do that how can I index my data so that I can use it when I want to retrieve related docs ?

appreciate your help all
Regards,

[mayankjobanputra]

Hey @shahad2099,
If I understand you correctly, your issue is that your indexed documents die whenever Colab runtime dies and you somehow don't want to reindex all these documents.

For this issue, I suggest 2 different solutions.

1. Use cloud stores like pinecone or AWS OpenSearch to store and index your data and you can query those whenever you would like.
2. You can use FAISS Document store to index, and download the index file and every time you want to query you can upload it to Colab and use it to query your documents.

[shahad2099]

Thank you for you'r answer ,
one more question , if I use FAISS then I can download the index files and work locally ? I mean I don't need to any other storage methods ? Because using ES is slow and not efficient as i expected .

Regards,