Milvus 2 with PostgreSQL

[dgen]

We are using Milvus2DocumentStore with a dedicated aws rds instance for PostgreSQL with the following specs:

RDS - Postgres v13.7
db.m6g.large
Provisioned IOPS - 1000 IOPS

We are getting this error quite frequently when the SQLstore tries to get the corresponding passages:

sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) SSL connection has been closed unexpectedly

is there any way around this?

[bogdankostic]

Hi @dgen!
I'm not familiar with AWS RDS but let me try to help. Do you see any other logs that indicate that something is not going as expected? In which cases do you observe this error?

[dgen]

Hi @bogdankostic,

When it happens, it is always during this call:

haystack/haystack/document_stores/milvus2.py

Line 442 in 3319ef6

documents = self.get_documents_by_vector_ids(vector_ids_for_query, index=index)

and it also prints the SQL query which is along the lines of :
[SQL: SELECT document.id AS document_id, document.created_at AS document_created_at, document.updated_at AS document_updated_at, document.content AS document_content, document.content_type AS document_content_type, document.index AS document_index, document.vector_id AS document_vector_id, meta_document_1.id AS meta_document_1_id, meta_document_1.created_at AS meta_document_1_created_at, meta_document_1.updated_at AS meta_document_1_updated_at, meta_document_1.name AS meta_document_1_name, meta_document_1.value AS meta_document_1_value, meta_document_1.document_id AS meta_document_1_document_id, meta_document_1.document_index AS meta_document_1_document_index WHERE document.vector_id IN (%(vector_id_1_1)s, ...) AND document.index = %(index_1)s [parameters:{...}]

It then usually leads to this:
sqlalchemy.exc.PendingRollbackError: Can't reconnect until invalid transaction is rolled back.
from which point all subsequent calls fail.

Thanks for your help