Google Docs-style annotation collaboration

[normanrz]

This would be cool for collaborating on skeleton annotations. CRDTs such as automerge could be used for that.

/edit:
By now, we have drafted the following milestones. For details, see this design doc.

• Milestone 1: Store User-dependent information separately
• Milestone 2: Live Updates for read-only users #8664
• Milestone 3: Proofreading (without segment list) #8724
• Milestone 4: Proofreading with segment list (but without undo)
• Milestone 5: Proofreading with segment list with undo
• Milestone 6: Volume annotation (with undo)
• Milestone 7: Skeleton annotation (with undo)
• Milestone 8: Bells and whistles 🔔
• Follow ups / Related issues:
  • Store layer visibilities in an annotation per user #8684
  • Annotation Layer Ordering is Inconsistent #8673 (affects consistency on download and re-upload)

[daniel-wer]

We prototyped some of this in a recent spike, the code can be found on the live-is-live branch.

What we did

• Frontend initiates websocket request to a backend endpoint when opening a tracing
• The backend keeps track of opened websockets
• Update actions sent by the frontend are mirrored to the other open websockets
• Update actions received via websocket are applied by the frontend, but are excluded from diffing (so they are not resent to the backend again)
  • Only createTree, createNode, and createEdge are applied for now
  • updateTracing update actions are used to show a circle where the editPosition of other tracers is (like a cursor in Google Docs)

What is missing

• Handling of the remaining update actions in the frontend
• Use of a CRDT such as automerge to allow for concurrent updates
• Ideally when opening the annotation of another user, it should not be editable directly, but there should be a button which can be selected to start editing
• Backend should track whether websockets are closed

What problems did we encounter

• Backend could only sent the update actions to all opened websockets, but it should actually select only the ones which belong to the same annotation (@youri-k you found something here, right? Could you edit in a link to the solution?)
• Backend and frontend maintain a synchronized version of the tracing. How would this work if a CRDT is used in the frontend - would the backend need to use the same CRDT?
• The editPosition, activeNodeId, etc. is only saved once per tracing, but would need to be saved per user of the tracing now.

Demo

The 3D view on the left is in one opened instance of the annotation whereas the right half of the image shows another opened instance. The changes are synchronized and the tracing can be edited in both opened instances.

References

https://www.dotconferences.com/2019/12/james-long-crdts-for-mortals https://youtu.be/DEcwa68f-jY?si=YijToM_g51IfdmBg
https://github.com/automerge/automerge

[MichaelBuessemeyer]

https://www.youtube.com/watch?v=0d73MdQkJOk might also be interesting