Plans for AIFS-ENS in API?

[Tristanjmeyers]

Hello!

I see that AIFS-ENS is going to some public datasets now (e.g. s3://ecmwf-forecasts/aifs-ens_testdata). Are there plans to make the AIFS-ENS available through this API once it's stable?

Thanks for your efforts.

[awarde96]

Hello, yes we have made a new release that should allow access to the aifs-ens test data. The notebook here https://github.com/ecmwf/ecmwf-opendata/blob/main/examples/aifs-ens-earthkit.ipynb should explain how to retrieve the data.

Once the aifs ensemble is operational it will be available via the standard source=ecmwf.

[Tristanjmeyers]

Thank you so much! I appreciate the reply and example notebook. I will check it out.

While I have you, I was wondering if you or your team have any plans to continue supporting and updating this API through upcoming release of ECMWF open data scheduled for October this year? It would be great to know if integration or compatibility is being considered.

[Tristanjmeyers]

I tried the notebook that you linked to, I wasn't able to set the source like you provided. I did add a .ecmwf-opendata file to my $HOME directory (and project root directory just to make sure) which had:

{
        "ecmwf-testdata" : "https://data.ecmwf.int/forecasts/aifs-ens_testdata/"
}

But I got an earthkit error ValueError: Unknown source 'ecmwf-testdata'. I'm on earthkit-data-0.15.0.

However, I got it to work simply by just adding the url to the request, e.g. :

ds = earthkit.data.from_source(
    "ecmwf-open-data", 
    source="https://data.ecmwf.int/forecasts/aifs-ens_testdata/",
    model="aifs-ens",    
    time=0,
    stream="enfo",
    type="cf",
    param=["msl", "2t"],
    target="data.grib2",
)

[MaartjeKuilman]

In October, users will not pay information cost anymore for their real-time data in dissemination, but there will not be any change to how the open data will be delivered.

[awarde96]

I am unable to replicate the error on my side unfortunately, and your version of earthkit-data looks ok.

Could you try with the client directly and see if you still get the same error?

from ecmwf.opendata import Client

client = Client(source="ecmwf-testdata", model="aifs-ens")

client.retrieve(
    time=0,
    stream="enfo",
    type="pf",
    param="msl",
    target="data.grib2",
)