multi-database access for reading within a changeSet

[ashishkujoy]

In our project we are using mongock for migration, till now in every migration we needed to connect with only one mongo data base but now we have use case where we want to access two mongo database from a single changeset. We went through mongock documentation but could not find any example of using multiple database.

Below is the code snippet of how we are currently creating a mongock runner

val mongoClient = MongoClients.create("mongodb://dummyuser:dummypassword@localhost:27017/databaseNameA")

MongockStandalone.builder()
            .setDriver(MongoSync4Driver.withDefaultLock(mongoClient, migratorConfig.databaseName))
            .addChangeLogsScanPackage("org.my.package")
            .overrideAnnoatationProcessor(ConditionalOnPropertyAnnotationProcessor())
            .buildRunner()

@ChangeSet(order = "01", id = "testId", author = "Team")
    fun modifyLeadRequestForABHI(db: MongoDatabase) {
         // migration logic here
    }

What we looking forward.

@ChangeSet(order = "01", id = "testId", author = "Team")
    fun modifyLeadRequestForABHI(@Name("databaseNameA") db1: MongoDatabase,@Name("databaseNameB") db2: MonogDatabase) {
         
    }

Let us know if something like this is possible with mongock.

[dieppa]

Hello @ashishkujoy ,

as part of the version 5, which we are working on, we are providing a multi-tenant feature. Although we haven't finished the design yet, our idea is to make Mongock able to support multiple MongoClients and apply all the changeLogs(or a subset by filtering) to all the mongoClients.

However, from your description, that's not exactly what you mean. You are trying to connect to multiples databases in the same changeSet. I am wondering if you really need that or the multi-tenant feature I just described would work for you.

If not, could you please provide some more context to understand your scenario and why the need of accessing to two(could be N) databases in the same changeSet?

Take into account that a changeLog/changeSet is linked to a single database, it wouldn't make sense to write to multiple databases in the same changeSet. A bit more sense, could be if you have the second databases just to read, but the actual changes are only to one database.

As said, please share you scenario and we'll see the best solution 😃

[ashishkujoy]

Thanks @dieppa for a quick reply.

In the change set we want to read a field from a collectionA of database A and add that field in another collectionB of database B.
So multi-tenant is not what we are looking for.

[dieppa]

Hi @ashishkujoy ,

ok, so I understand you have a unique target database, where you will be writing to and a second database where you will be reading from. That makes more sense!

I need to discuss it within the team, but I am pretty confident we will be adding that to the version 5. I think it's an interesting one!! Although the expected release date for the version 5 is by the end of June, as earliest, we can provide a beta version for you to start using and checking it. Please share your contact by sending us an email to dev@cloudyrock.io, so we can keep you on the loop.

Alternatively and regardless the addition of the feature you suggest, we are also providing the CLI as part of the version 5(it may come a bit later than the version 5 release though), which will provide the feature diff. Diff command will compare two databases and generate the changeLogs to get one database to the state to the second one. As said, it's not that flexible and powerful and it would complementary, but I am wondering if that would work for you in this case.

Regards

[ashishkujoy]

Thanks @dieppa I sended an email to above mentioned email id.

[vbotteman]

Hi @dieppa ,

In our setup we have one MongoDB instance with a database per customer. Each database has for example an event collection. Can we migrate the event collection in all the databases with one script using the multi-tenant feature?
Where can I find more info about the multi-tenant feature?

Regards

[dieppa]

Hello @vbotteman ,

in short, yes, you can do a single mongock migration and apply it to all your client databases by using the multi-tenant feature.

We haven't provided documentation for this because it's still in beta, but it we'll do it soon.

Are you using Spring boot?

[vbotteman]

Hello @dieppa ,

We use Spring Boot 2.5 and Java 11.

[dieppa]

Ok. So currently multi-tenant feature doesn't work with spring data repositories neither @EnableMongock. You need to use Mongock builder and MongoTemplate directly.

We are still working on this and will provide the proper documentation. Meanwhile you can get more information in our multi-tenant example project.

Let us know how it goes :)

[RoniBam]

Hello,
I'm also facing a scenario where in one changeUnit I need to read from Database A and write to Database B. Is this option currently supported by mongock version 5?
thanks

[osantana85]

Hello @RoniBam , for that scenario, please take a look at this Mongock's sample project that uses a secondary DB to retrieve data:

Mongock - MongoDB with Springboot (Advanced)

Note that this sample project is not using multi-tenancy feature, it only shows you how to use a secondary DB connection (for reading purpose) in Mongock's ChangeUnits.

Hope this helps.