Focus area: RDM tool interoperability for samples

[PeterKraus]

Copied from the #1, this discussion is to establish a knowledge-base in the area of RDM tools and their interoperability with focus on samples, as part of the MADICES-2024 workshop:

• Focus on exchanging sample information (metadata) between RDM softwares (i.e. machine-machine, not user-machine)
• Building on the agreed infrastructure solution above.
• Annotating samples - mutable/immutable, consumed/available, computational/experimental
• Dealing with exchange of sample history

Please include links, example sample exchange data, and discuss approaches and necessary metadata below.

[simontaurus]

I think this could build on https://www.researchobject.org/ro-crate/, a resource self-describing graph in JSON-LD, e. g. used to exchange ZIP dumps between RDM/ELN tools: https://github.com/TheELNConsortium/TheELNFileFormat/blob/master/SPECIFICATION.md

[PeterKraus]

Annotating sample history conventions

I would hope we could arrive at a consensus for annotating or indicating different "sample history conventions" that are common in people's data workflows:

• sample-centric workflows, where experimental data gets attached (sequentially or by timestamp) to the same sample entry
• process-centric workflows, where an experimental process has a sample as an input and a (new instance of a) sample as an output

There might be hybrids between these two extremes. I don't want to spend time on the discussion of the merits of either, but for RDM tools it would be helpful if the sample object in an ELN had an "annotation" indicating which of these conventions applies.

[simontaurus]

The good thing about graph representation of data is that we do not have to make a decission on that, because
Process -makesStatement-> Measurement
Measurement -hasSubject-> Sample

On the other hand, in practise and object oriented views we have to make a decission if the measurent is part of the process, part of the sample or is a standalone entity. IMHO, during data generation it is convinient to make measurement within the process documentation but later on we should provide a view (and data export) on all measurements from the sample perspective.

[PeterKraus]

Linking to out-of-band data

In an ideal world, the LIMS would contain information about all "samples" (including chemicals, instruments, etc.) and the ELN would be the single source of truth for their history, relationships, and data.

In practice, it might be useful to keep some data stored locally, in the automation tool or WFMS, to both avoid data duplication and unnecessary network traffic.

A mechanism by which it's possible to:

• indicate linked data exists
• indicate how it can be accessed
• indicate what kind of data it is

would make it possible to keep the data where it's needed the most, and still allow us to ensure that when the sample is "archived" (published or exported), the linked data is fetched and included in the archive. This would go a long way towards interoperability.

[simontaurus]

I think this would be perfectly possible as long as all acting parties are make additional data available (public hosted, maybe access restricted, e.g. S3 Object store or Triple store). With the Semantic Web and IRIs we already have the foundation for distributed data. There are also developed data space concepts like EDC and SolidPods to address this issue

[edan-bainglass]

From the Goal statement discussion:

• do we need to enforce a certain ontology for sample management, e.g. should samples be allowed to be stateful or restricted to be immutable?

I understand stateful samples to be immutable, each carrying information on its state. So when a "change" happens, a new sample object is created with a new state. Perhaps there are two objects, one permanent Sample containing the immutable parts (mostly metadata), and a State object (subclass) that contains the current state.

Thoughts?

[edan-bainglass]

Take coin cell data, for example - though valrious amounts change, the general composition does not, so electrodes/electrolyte formulas, separator/spacer/case types, raw data units, user input (various descriptions, including assembly info, batch/sub-batch info).

[kjappelbaum]

the general composition does not, so electrodes/electrolyte formulas

i think this is often a good example, but not always. What do you do if stuff degrades/decomposes/reacts?

[edan-bainglass]

Sure, in that case, there will be less surviving metadata, but still some, no? Manufacturing info for example (where, when, who. how, etc.).

[ml-evs]

The EMMO README (https://github.com/emmo-repo/EMMO) provides some of the philosophical grounding to this (or at least, in my case, gives me fancy words to look up on wikipedia). I think the best way of discussing this is to come up with a contrived example of recording the same set of data/measurements/samples with these different philosophies (e.g., sample-driven, event/measurement-driven, process-driven). My feeling is that the interoperability layer might need to impute a temporal basis to make things work, e.g., in the case of something like ELN file format, we might need to implicitly make a sample state per measurement and faithfully representing any uncertainty in terms of ordering of measurements.

To be concrete, say I am studying degradation of a coin cell and I measure some diffraction pattern of my starting electrode, then assemble the coin cell, cycle it for 6 months, then deconstruct it and take another diffraction pattern. Using something like the ELN file format I either represent this in my zip file as have 1 entry with two measurements, two (disconnected?) entries with a measurement each, or maybe even 1 sample with 1 measurement file containing two diffraction patterns.

[simontaurus]

This can be addressed by having a process centric view, so we can say

FreshCoinCell -isInputOf-> DegradationProcess -hasOutput-> DegradatedCoinCell
(means DegradatedCoinCell -isSuccessorOf-> FreshCoinCell)
FreshCoinCell -isInputOf-> DiffractionMeasurementProcess -hasOutput-> DiffractionPattern1
DegradatedCoinCell-isInputOf-> DiffractionMeasurementProcess -hasOutput-> DiffractionPattern2
This pattern makes sense if we have a major change between the states or even (diss)assemble objects.

Modelling with states would require some subgraphs with time-limited validity so we know when (at latest right after the DegradationProcess) our single CoinCell object changes its state from "fresh" to "degradated". This makes sense if the change is regarded as minor so we do not endup with billions of slightly different objects.

However, in both cases exporting all process steps and subgraphs addressing a sample would provide a proper self-description to understand the data correctly