InMemory document store

[chaitanya-nit]

Hi All,

I want to understand why InMemory document store is not recommended for production workloads? Can anyone elaborate on this?
Is this not recommended for large number of document search? or any particular thing?

[anakin87]

Hello!

InMemoryDocumentStore is not recommended for production workloads for several reasons:

• it lives only in memory, so you can't save or reload it
• its size is conditioned by the resources available in your machine
• it is based on simple Python data structures, which are not optimized for fast writing and retrieval
• speaking of dense retrieval, it can't apply Approximate Nearest Neighbours (ANN) at query time.
  Each time a query is made, it scans each document in the Document Store to compute the similarity to the query.

So, InMemoryDocumentStore is good for experimenting and does not require any particular setup.
Depending on your use case (sparse or dense retrieval), there more efficient and production-ready solutions, as you can read in the documentation.