State of the art of collaborative online editing

[da5nsy]

Hi folks,

I'm curious what people think is the state of the art for online collaborative editting of research communication documents (often/historically referred to as "journal articles").

What I'm seeing:

• There's Overleaf for LaTeX editting
• There's stencila for [something? it's not clear what the underlying structure is] (links: https://stencila.io/ https://github.com/stencila/stencila) (and it's unclear where they get their money or what their sustainability plan is)
• There's the old-school way of git and then some online forge with CI that could build and output a thing (but this doesn't have the "live editor" functionality - unless: is there a way to hack GitHub codespaces into running as a content server?)
• There's collaborative markdown pads (etherpad, hedgedoc, cryptpad) etc, but that requires someone to manually download and process the files.

Challenges:

• It kind of needs to be pay-to-play (or at least freemium) because there's arbitrary compute potential.

Options:

• I feel like Overleaf could (relatively) click their fingers and add quarto and jupyterbook support and they'd sweep up, but my impression (warning: spicy and uninformed opinion incoming) is that Digital Science see Overleaf as a cash cow and are happy to let it stagnate from a tech perspective.
• Is this something that Binder would be well-placed to support - they have experience with the "we want to make things reproducible/collaborative but also compute is expensive" landscape, and I feel like going to a Github repo and clicking a launch in Binder (or something like that workflow, which allows multiple people to connect to the same one) would be a just-about-palatable workflow for even people who are relatively code-shy.

The fact that the mystmd folks are writing papers in Google Docs makes me think that the answer is "there's nothing good out there" but keen to hear where people think this is up to and where we need to push/create for this to move forwards! ✨

See also: discussion on the quarto discussion section: quarto-dev/quarto-cli#405

[fecet]

AFAK the best solution for markdown (and quarto) is still jupyter collaboration

[choldgraf]

I think that technology projects like JupyterHub and Binder could naturally enable this kind of thing in a more community-centric way rather than relying on a single SaaS provider. That said, the tech isn't quite there yet. JupyterHub tends to focus on individual servers currently, rather than collaborative ones with shared storage. That said, I think the JupyterHub team is interested in enabling more collaborative workflows in general, and I bet that tools like Real Time Collaboration in JupyterLab are things we could build on top of as well - that'd give you multi-user document editing which would be a great first step forward.

So in short, I think JupyterHub and Binder could definitely enable things like this, it would require some development work to facilitate and probably would need to be a funded effort rather than something that'd be tackled with bits and pieces of people's time.

[da5nsy]

Awesome, thanks @choldgraf. That all makes sense.

If there are things you think we could/should be doing towards this effort - I'm keen to help.

I guess the first step might be documenting what is currently possibly and mapping out what would need to happen to move forwards?

[choldgraf]

I think having documented experiments that both:

• Validate the value by showing that people are trying to do this, and generally like the outcomes.
• Show what's possible right now that we can just document better, vs. what will need more development

Would be really interesting and helpful!

[da5nsy]

It might be fun to have a bit of a hack session with the prompt of "if we were writing a paper (with some computational) elements in it together, what could we do right now? Where does it break/need work?"
Happy to be part of that.