storing sensitive information on Vector DB

[vishyarjun]

Hi All,

We have recently asked by our customers not to store the customer data in vector databases as they are already stored in RDBMS.
As of now we are using vector databases to store distinct cardinality columns that can enhance the search. we are using retreiver tool like below

retriever_tool = create_retriever_tool( retriever, name="search_proper_nouns", description=description, )

by not storing the proper nouns, we might be losing accuracy. what should we do in this situation?

Also another challenge is that this data is growing, do we have to embed data in a schedule to update the new data that gets added everyday.

Cheers,
Arjun.

[maximeperrindev]

Hi @vishyarjun,

Here is something very cool about data anonymization using a local model : https://python.langchain.com/v0.1/docs/guides/productionization/safety/presidio_data_anonymization/

I think that could help you