How to add your own custom DocumentStore

[sgaseretto]

Is there any documentation on how to add your own DocumentStore? Like if for example someone wants to add Weaviate or Vearch to store de representations from the Documents or something like that.

[julian-risch]

Hi @sgaseretto a WeaviateDocumentstore is already available in Haystack: https://github.com/deepset-ai/haystack/blob/master/haystack/document_stores/weaviate.py
If you are thinking about implementing a custom Documentstore you can have a look at the base class to see the required methods: https://github.com/deepset-ai/haystack/blob/master/haystack/document_stores/base.py but there is no additional documentation on how to implement a custom document store. I think most users should be able to find everything they need in the currently available document stores. Out of curiosity: are there any particular features that you are currently missing?

[sgaseretto]

Hi @julian-risch, thanks for the fast response! I'm currently working with a Vector Search Engine similar to Weaviate called Qdrant, and would like to use it with haystack. I will experiment with the BaseDocument and see if I can I add it to Haystack. Thanks for the links!

[rvstraalen]

Hi @sgaseretto, I was wondering if you got this working and if we might expect this to be added to the haystack repo. I'm also looking into using Haystack with a Qdrant document store.

[anakin87]

Hello!

Probably the Qdrant team is working on an integration.
See https://github.com/qdrant/qdrant-haystack

I also tag @kacperlukawski who is working on it...

[kacperlukawski]

@sgaseretto Yeah, we're working on that. Hopefully, we can make some progress this week! If you'd like to participate, I may invite you to a project.

[kacperlukawski]

@sgaseretto @rvstraalen The first (beta) version has been released to PyPI repositories. If you want to start using Qdrant right away, this is the best way, as this is an official package we're going to support: https://pypi.org/project/qdrant-haystack/

If you're interested in contributing, here are the sources: https://github.com/qdrant/qdrant-haystack/

I'll appreciate any feedback!