Question about requirement for original stack for CorrectedImageStack

[ehennestad]

Hi,

It seems that the original property is needed for the CorrectedImageStack type. I would argue that in many cases it is undesirable to add the original two photon data in the NWB file 1) because it can take up a lot of data space and 2) because the utility of this is very low as common motion correction algorithms are good at stabilising images and represent the original data accurately.

What are thoughts on this? Are there known workarounds, like adding a link the an external two-photon series with a pathname that is relative to system that generates the nob file, but not resolvable by anyone else?

[CodyCBakerPhD]

The current motion correct types are something we've intended to revamp for a while

because the utility of this is very low as common motion correction algorithms are good at stabilising images and represent the original data accurately. Are there known workarounds

The primary workaround for this at the moment is to just not annotate the corrected image series using the CorrectedImageStack type (which exists to relate an 'original' ImageSeries with an existing 'corrected' ImageSeries, both of which assumed to be exist in the same file)

Instead, simply do not write the 'original' ImageSeries in the same file (perhaps save to a different file for provenance, a kind reminder that the DANDI archive will take files up to 5 TB in size free of charge), and only include the corrected series as the primary ImageSeries of the file

like adding a link the an external two-photon series with a pathname that is relative to system that generates the nwb file

Yes, that workaround has been used to create NWB files before, however

but not resolvable by anyone else?

for this reason and some others, the DANDI archive does not currently support external links of this form

[ehennestad]

So if I understand you correctly, I can take my motion corrected data and add it as a regular TwoPhotonSeries type? Are there any recommendations on where to put it? Should it then go to the acquisition group, or should it anyway go in an ophys processing module?

[CodyCBakerPhD]

So if I understand you correctly, I can take my motion corrected data and add it as a regular TwoPhotonSeries type?

Correct - the only missing piece might be details of how the correction was performed and any hyperparameters of it (such as x/y shifts?) which I would annotate well within the description of the series or if heavy enough, as an additional DynamicTable perhaps (can't offer particular suggestion without knowing more details of how correction was performed)

Are there any recommendations on where to put it? Should it then go to the acquisition group, or should it anyway go in an ophys processing module?

It's entirely up to you on which module to put the series in

I would personally say that it depends on the story you want your data files to tell - if you put it in acquisition you are saying it is the 'raw-most' basis for all the other processing data in the file (such as ROI segmentations, DF/F, etc) - if you put it in processing then it declares it as being just another step in your pipeline that occurred after acquiring the raw (the original) imaging data