Create a to/from xarray method/function

[DanRyanIrish]

Description

Support of gWCS in ndcube enables the possibility of converting between an ndcube and xarray objects. Although ndcube and xarray will still be preferred in different situations, the ability oto convert between them may still be highly useful. Is this something ndcube should support?

[Cadair]

We had a very productive discussion with Martin Durant @martindurant and Jim Bednar at the sunpy coordination meeting today. I think that as far as WCS / xarray integration goals go from an ndcube perspective this issue is probably the best place to start.

As I mentioned in the discussion today I am very happy to bring the WCS expertise to any effort to build out xarray support for WCSes. I suggest we use this issue as a good place to coordinate this work and discussion with other interested parties.

Also relevant to this is @eteq's issue pydata/xarray#3620

[martindurant]

cc @jbednar , who will get in touch when his team is ready

[ianthomas23]

Hi all! I am the third member of the @jbednar and @martindurant trio. Sorry I couldn't attend the discussion a couple of weeks ago, but I have watched the video of the conversation.

I am going to be at the NumFOCUS Project Summit in 2 weeks and hope to have a face-to-face chat with @wtbarnes. Hopefully we can involve the xarray folks too. I am happy to chat remotely before that if it helps!

[Cadair]

hey @ianthomas23 👋

Unfortunately I wont be at the NumFOCUS summit, but I am happy to chat either before or afterwards. I am super keen to see how far we can get with WCS + xarray and how to integrate it into ndcube.

Do you have a plan for what you are going to be working on yet?

[ianthomas23]

Do you have a plan for what you are going to be working on yet?

I think we have to have a pragmatic 2-stage approach. Firstly write some experimental code using WCS (astropy.wcs?) that is reasonably correct and reasonably performant so that we can progress with the rest of our project. This gives us a bit more time and space for stage 2 which is to find the correct long-term solution. Exactly what stage 2 looks like has yet to be determined, as does where the solution lies and who works on it!

We are hoping to talk to some xarray devs next week sometime to get their latest updates. Maybe we should have a chat after that?

[benbovy]

We are hoping to talk to some xarray devs next week sometime to get their latest updates. Maybe we should have a chat after that?

I'd be interested to join the discussion. We've made progress on Xarray custom indexes recently and hopefully it should be ready for "public" use (although still very experimental) with the next release. I'm happy to present about the latest updates on this front. I'd also like to allocate some time on working on or helping domain experts experimenting with custom indexes. From what I understand in pydata/xarray#3620, WCS + xarray looks like a nice fit for testing this feature!

[wtbarnes]

Hi @ianthomas23. Great to hear you'll be at the NumFocus Summit next week. Though I'll be the only SunPy maintainer present, I believe there will also be several astropy maintainers there in person as well who may be interested in this discussion or who can at least provide some WCS expertise.

[ianthomas23]

Following the in-person meeting last week at the NumFOCUS Project Summit, I will progress with some standalone code to go from FITS file to Xarray using flexible indices to generate WCS coordinates using the AstroPy WCS module. This will probably only support the subset of WCS that our current project requires. Other meeting attendees to provide guidance: @eteq (AstroPy), @wtbarnes and @Cadair (SunPy), @andersy005 (Xarray).

The subsequent longer-term goal is that this code will become part of wider ndcube/Xarray interoperability.

[ianthomas23]

Here is some progress to report: https://nbviewer.org/gist/ianthomas23/bbf85d1a38f8f161a2c1f641d9465096.

This is a Jupyter Notebook that uses an xarray.Accessor to convert from pixel to world coordinates using astropy transforms. It follows the approach of pydata/xarray#3620 (comment) in using a pixel_to_world accessor function that returns a new xarray.Dataset with WCS coords. The next comment in that issue describes an alternative approach that combines both pixel and world coordinates in the same Dataset.

This is pretty much everything that I need in the short term. It is quite specific to my test data and will need generalising to other use cases. I have played around a bit with a functional index (xarray.core.indexes.Index) a little as well, to allow e.g. slicing of pixel indexes based on WCS values, but I don't have anything concrete from that as I don't think I need it.

Probably not strictly relevant to target audience here, but if you download and run the notebook locally you can interactively zoom and pan the image using the mouse and controls on the right.