Synchronization on unstable connection

[matevzcesar]

Hello.

We're working on an app, where devices are expected to sync in remote areas with an unstable connection. We were planning on using the DMS HTTP mode. My question is, how does DMS handle situations, where the connection is lost mid-sync? I presume that all partial data is rejected and the sync continues from the last uninterrupted sync? In that case, would it be possible for us to implement a streaming API without modifying the library?

Thanks!

[Mimetis]

Everything is encapsulated in transactions, so you should be fine.
For streaming, I don't think it's possible, without major changes
But I assume you can do a pulling system from your devices

[matevzcesar]

Hi Sébastien, thank you for responding!

Could you elaborate on what you mean by a "pulling system"? I noticed in the docs the "Batch mode", which seemed perfect for our purpose, but after testing it out I noticed that in case of network interruption mid-sync, it does not finish the sync with partial data. I ended up with a schema, but no data. I guess batch mode is not meant to work in this way?
Also how does the batch size affect the functionality? Based on this parameter the sync sometimes finishes without errors and sometimes it just hangs, mostly when the batch size is smaller. Even running the sync twice with same batch size can produce different results.

[Mimetis]

Well, make a timer, and synchronize every 10 minutes maybe ?

Batch is here to download a sync json file in multiple chunks, nothing related with any stream technic
Imagine you need to synchronize let's say 100K rows, representing 10Mo (in Json)
It could work, but you may encounter some timeout from your users, especially if they have a low bandwith.

BatchSize will create chunk of approximatively 1 mo (if you set BatchSize to 1000) and will sent 10 http file of 1 MO, instead of 1 file of 10 MO
You users will not have a timeout in that case

Make sense ?

[matevzcesar]

Yes, thank you!