Proprietary Datasets

[elbertmarda]

Working group notes and discussion in Berlin

[edan-bainglass]

@PeterKraus, @ml-evs could you please summarize discussion/results here? 🙏

[PeterKraus]

In this WG, we had two "group discussion" sessions on Tuesday, and mostly pairwise discussions on Wednesday.

Discussion summary

The first discussion on Tuesday morning focused on approaches we use to gather instrument data, and involved @SteffenBrinckmann, @nicolejung, @elbertmarda, @CorsinBattaglia, @sanbrock, @esdal1, @PeterKraus - I hope I didn't forget anybody.

• @elbertmarda introduced OpenMSIstream as a way to make up-to-date data snapshots available downstream by leveraging streaming data
• @CorsinBattaglia and @PeterKraus discussed how tomato accesses instruments and returns data
• @nicolejung presented the system Chemotion uses to pick-up experimental data and ingest it into their ELN
• @SteffenBrinckmann showed us what ingesting the example MADICES2 datasets looks like in PASTA-ELN

My takeaways from this discussion was:

• Everybody has their own way of structuring their CSV files (or even several). Trying to make a library that understands everything seems hopeless. However, there is certainly scope for collecting "magic bits" (i.e. identifying information) about such files.
• It is important to try and push instrument vendors to implement open and standardised formats that are well annotated. This often doesn't work on a pairwise basis (a single lab buying custom devices from a big vendor) even in good business relationships (intentionally not naming names here). However, a concerted push from the community might get us somewhere, especially if backed e.g. by national funding bodies.
• Pushing directly from the instrument into an "ELN" without intermediate (local) storage is not a good idea. In hindsight this might be obvious for technical reasons, but happy to have an agreement among the few of us here. However, discouraging users from accessing the intermediate storage is important, as the "ELN" should be the point-of-access for user data. Approaches here vary greatly.

In the Tuesday afternoon discussion, we were joined by @dalito and @nmoust presenting their work on Ontologisation in Catalysis as part of NFDI4cat's voc4cat:

• While a lot of work has been done on surveying what is available for catalysis, it is very early days when compared to e.g. BATTINFO.
• The voc4cat effort is really tailored to facilitate contributions from domain experts (via an Excel-based workflow). This is commendable and might be a good way to get a broader community involved.

Outcomes

• Identified the need to provide datasets with at least some semantic annotation in case of a future workshop. It is my understanding, that the most used dataset was the catalysis dataset contributed by @schumannj, in particular the archive passed via NOMAD. This means ontologisation/semantic annotation of data should happen ideally at the point of data creation.
• Incorporated PASTA Converters into the Datatractor yard with the help of @PeterKraus, @SteffenBrinckmann, and @ml-evs
• Identified scope for contributing Chemotion Converter readers into the yard with @nicolejung and @StarmanMartin.
• We attempted to semantically annotate the Robocasting dataset with @esdal1. With the help of @simontaurus, and later @dalito and @sanbrock, we have basically arrived at the same conclusion as shown in @samikanza's example here: a manual annotation of a representative dataset using a combination of compatible ontologies (including voc4cat as appropriate) is necessary first, but can hopefully be made automatic if incorporated into tooling.

I'll let others include what they might want to have documented for the Wednesday sessions.