Need an endpoint for getting the latest sequence ID

[bardram]

Right now sequence IDs are enforced - see #375 .

If this is the case, we need a way to get the latest sequence ID for a data stream, so that we can continue from there (and avoid the error).

[Whathecode]

Yup. This is missing information which most likely belongs in deployment status.

When you redeploy a device, you need to know from which sequence ID to start uploading data.

In normal operations, though, it is the client who locally track that state, as it is the client who determines sequence IDs.

[bardram]

We should make the sequence ID optional when uploading.

[Whathecode]

I seem to recall we discussed that option in the past. Do you remember the pro/cons? There are very good design reasons for it being there (dedecting dropped data, handling out of order uploads)

If it becomes optional, that should likely be configurable on a deployment, and consequently data streams level. It shouldn't be optional by default for all deployments.

It's also not really related to this issue. The last ID will still need to be exposed when you do want to specify sequence IDs, as you need it when re-deploying a study to a client.

I suggest opening a new issue if you want to stick with making it optional. But, what is the main problem? Don't you have client-side persistence to keep track of this?

[bardram]

I seem to recall we discussed that option in the past. Do you remember the pro/cons? There are very good design reasons for it being there (dedecting dropped data, handling out of order uploads)

Yes - and we keep the sequence ID, but do not require it.

The problem is that we don't use it, and the fact that it is mandatory makes it impossible to upload any data. We had to make a serious hack i the CAWS to circumvent this.

We rely on timestamps.

If it becomes optional, that should likely be configurable on a deployment, and consequently data streams level. It shouldn't be optional by default for all deployments.

No need for this convoluted solution. Users uploading data can just use it if they want (we don't).

It's also not really related to this issue. The last ID will still need to be exposed when you do want to specify sequence IDs, as you need it when re-deploying a study to a client.
I suggest opening a new issue if you want to stick with making it optional.

Yes - creating a new issue. @NGrech + @yuanchen233 - include this in your 1.3 release.

But, what is the main problem? Don't you have client-side persistence to keep track of this?

The problem is that client-side apps get deleted with all persistence regularly (on re-install). So - with this new endpoint, we can get the latest sequence ID. But the whole point is that we don't need this at all, and we can't come up with any scenario where we don't just rely on the timestamps on the data.