On multiple Contexts, chat with IPSL

[mo-marqh]

---

From: xios-users xios-users-bounces@forge.ipsl.jussieu.fr on behalf of Mark Hedley mark.hedley@metoffice.gov.uk
Sent: 06 February 2024 17:09
To: xios-users@forge.ipsl.jussieu.fr
Subject: Re: [XIOS-users] XIOS read into domain

Hi Julien

this makes much more sense, thank you for clarifying this for me

I now understand this pattern and how to implement it effectively. it is indeed useful

merci beaucoup
mark

---

From: Julien Derouillat julien.derouillat@lsce.ipsl.fr
Sent: 06 February 2024 15:40
To: xios-users@forge.ipsl.jussieu.fr
Cc: Mark Hedley
Subject: Re: [XIOS-users] XIOS read into domain

This email was received from an external source. Always check sender details, links & attachments.

Hi Mark,

You can have many contexts defined simultaneously, this is for instance the case on our side, where on a MPI process contexts are defined for : the dynamical core, the atmosphere and the land surface models.
Try to explicit the switch from a context to another using :

call xios_context_initialize('axis_check', comm)
! ... ...
call xios_close_context_definition()

call xios_context_initialize('main', comm)
call xios_set_current_context('main')
! ... ...
call xios_close_context_definition()

Regards

Julien

Le 06/02/2024 à 14:58, Mark Hedley a écrit :

many thanks for the further advice Yann

i have implemented a first-pass context that gives access to configuration information, then a second context that is configured from the first, and then used.

This pattern makes sense, i can see its utility. i now have working and sensible code.

there is one facet, which caught me out for a bit, and has left me thinking about how to apply this pattern in more general cases.

initially i implemented

    call xios_context_initialize('axis_check', comm)
    ! ... ...
    call xios_close_context_definition()

    call xios_context_initialize('main', comm)
    ! ... ...
    call xios_close_context_definition()

but this programme does not run, it hangs at the second call xios_close_context_definition()

i needed to call xios_context_finalize() between the two context initialisations, as per

    call xios_context_initialize('axis_check', comm)
    ! ... ...
    call xios_close_context_definition()

    ! ensure to finalize this checking context before initialising
    ! the main context for further use
    call xios_context_finalize()

    call xios_context_initialize('main', comm)
    ! ... ...
    call xios_close_context_definition()

This works, but i am not sure I fully understand why this is needed, and the implications for managing multiple contexts.

Is there any further insight you would share with me about why it is necessary to call a context_finalize before the next context_initialize?
Is there an expectation on the number of contexts being open or in use at a particular stage in the programme? is this singular?

many thanks
mark

---

From: xios-users xios-users-bounces@forge.ipsl.jussieu.frmailto:xios-users-bounces@forge.ipsl.jussieu.fr on behalf of Yann Meurdesoifyann.meurdesoif@cea.frmailto:yann.meurdesoif@cea.fr yann.meurdesoif@lsce.ipsl.frmailto:yann.meurdesoif@lsce.ipsl.fr
Sent: 05 February 2024 17:37
To: xios-users@forge.ipsl.jussieu.frmailto:xios-users@forge.ipsl.jussieu.fr
Subject: Re: [XIOS-users] XIOS read into domain

This email was received from an external source. Always check sender details, links & attachments.

Hi Mark,

Le 05/02/2024 à 18:01, Mark Hedley a écrit :

Hello Julien

many thanks for the advice

i have a functioning programme, but i have fragile code, as i do not seem able to access the domain ni_glo nj_glo in order to enable dynamic setting of the ni nj ibegin jbegin

I can hard code values (but this is not great for the coding patterns), e.g.

    call xios_initialize('client', return_comm=comm)
    call xios_context_initialize('main', comm)

    ! ! Commented out, "Data is not initialized"
    ! call xios_get_domain_attr('original_domain', ni_glo=lenx)
    ! call xios_get_domain_attr('original_domain', nj_glo=leny)
    ! call xios_get_domain_attr('resampled_domain', ni_glo=lenrx)
    ! call xios_get_domain_attr('resampled_domain', nj_glo=lenry)

    ! hard coded to match input file
    lenx = 5
    leny = 5
    lenrx = 2
    lenry = 2

    call xios_set_domain_attr("original_domain", ni=lenx, nj=leny, ibegin=0, jbegin=0)
    call xios_set_domain_attr("resampled_domain", ni=lenrx, nj=lenry, ibegin=0, jbegin=0)

    call xios_close_context_definition()

but if i try to uncomment the code to obtain ni_glo nj_glo and dynamically assign ni nj
then i get
In file "type_impl.hpp", function "void xios::CType<T>::_checkEmpty() const [with T = int]", line 210 -> Data is not initialized

Are there ways to obtain access to ni_glo nj_glo from a read input file to enable dynamic setting of ni & nj within the context_definition?

i know that XIOS has knowledge of these values at this point, as i get errors if my hard coded values are not consistent; e.g.

lenx = 4
call xios_set_domain_attr("original_domain", ni_glo=lenx, ni=lenx, nj=leny, ibegin=0, jbegin=0)

returns

The domain original_domain has ni_glo read from file domain_input.nc and ni_glo provided from model are not coherent. They should be the same.
 ni_glo read from file: 5 ni_glo from model: ni_glo="4"

XIOS will analyse all attributs from XML or from model, and the inputs files when calling close context definition. So the attributes ni_glo, nj_glo, ni, nj, etc. will be available only after this point.

If you need to use these values to initialize some attributes on an XIOS context, you need to declare 2 xios context. A first one to read these values and a second to used these value (and eventually allocate array of the size) in your workflow. This method is used commonly in our models, a initial context is launched et closed in order to prepare and initialize the "good" context.

Note that if you need values related to your mpi rank (so value ni, nj, ibegin, jbegin...), you need to add a specific transformation on your domain :

<grid id="original_grid">
  <domain id="original_domain" type="rectilinear">
     <generate_rectilinear_domain/>
  </domain>
</grid>

So in this case, XIOS will automatically decompose your horizontal domain following your MPI distribution (each rank will be assigned to a specific 2d-piece of your domain).

Otherwise each MPI process will retrieve to global domain, which will work but will be not very performant since not distributed over process, ie ni=ni_ni_glo, ibegin=0, nj=nj_glo, jbegin=0 for all processes...

Best regards,

Yann

many thanks
mark

---

From: Julien Derouillat julien.derouillat@lsce.ipsl.frmailto:julien.derouillat@lsce.ipsl.frmailto:julien.derouillat@lsce.ipsl.frmailto:julien.derouillat@lsce.ipsl.fr
Sent: 02 February 2024 08:49
To: xios-users@forge.ipsl.jussieu.frmailto:xios-users@forge.ipsl.jussieu.frmailto:xios-users@forge.ipsl.jussieu.frmailto:xios-users@forge.ipsl.jussieu.fr
Cc: Mark Hedley
Subject: Re: [XIOS-users] XIOS read into domain

This email was received from an external source. Always check sender details, links & attachments.

XIOS is developed to be run in a parallel context, to read data consistently you have to provide the local informations (slide 32 of the talk) which describe the domain decomposition.

ibegin="0" ni="5" jbegin="0" nj="5"

Aside to the slides provided yesterday, you can find some hands-on. The hands-on 5 (data read, but no interpolation there, just a copy) could be a good start point, the source code is light (don't hesitate if you have question).

The repository of the hands on : forge.ipsl.jussieu.fr/ipsl/forge/projets/ioserver/svn/XIOS2/xios_training

Regards

Julien

Le 01/02/2024 à 17:48, Mark Hedley a écrit :

Hi Julien

many thanks for the input on this.

I think I understand that I want a domain, not a combination of 2 axis instances

I try to define two domain instances, to be read from the input netCDF file:

   <grid_definition>
    <grid id="original_grid">
      <domain id="original_domain" type="rectilinear"/>
    </grid>

    <grid id="resampled_grid">
      <domain id="resampled_domain" type="rectilinear"/>
    </grid>
  </grid_definition>

  <field_definition prec="8">
    <field id="edata" name="resample_data" grid_ref="resampled_grid" long_name="expected resampled data values" unit="1" />
    <field id="odata" name="original_data" grid_ref="original_grid" long_name="input data values" unit="1"  />
  </field_definition>

  <file_definition type="one_file">
    <file id="domain_input" output_freq="1ts" mode="read" enabled=".true.">
      <field id="odatain" name="original_data" grid_ref="original_grid" operation="instant" />
      <field id="edatain" name="resample_data" grid_ref="resampled_grid" operation="instant" />
    </file>

  </file_definition>

I seem to get coordinate values reported as read within the xios_client.err:

(9) **************** void xios::CDomain::fillInLonLat()
Object id="original_domain" object type="domain"
*** XIOS attributes as defined in XML file(s) or via Fortran interface:
[latvalue_rectilinear_read_from_file="(5) 0 ... 8" lonvalue_rectilinear_read_from_file="(5) 0 ... 8" name="original_domain" ni_glo="5" nj_glo="5" type="rectilinear" ]
*** Additional information:
[]

(10) **************** void xios::CDomain::fillInRectilinearLonLat()
Object id="original_domain" object type="domain"
*** XIOS attributes as defined in XML file(s) or via Fortran interface:
[latvalue_rectilinear_read_from_file="(5) 0 ... 8" lonvalue_rectilinear_read_from_file="(5) 0 ... 8" name="original_domain" ni_glo="5" nj_glo="5" type="rectilinear" ]

but something is not initialised:

In file "attribute_template_impl.hpp", function "void xios::CAttributeTemplate<T>::checkEmpty() const [with T = int]",  line 78 -> On checking attribute with id=ni : data is not initialized

I fear that I miss something obvious

thank you
mark

---

From: Julien Derouillat julien.derouillat@lsce.ipsl.frmailto:julien.derouillat@lsce.ipsl.frmailto:julien.derouillat@lsce.ipsl.frmailto:julien.derouillat@lsce.ipsl.frmailto:julien.derouillat@lsce.ipsl.frmailto:julien.derouillat@lsce.ipsl.frmailto:julien.derouillat@lsce.ipsl.frmailto:julien.derouillat@lsce.ipsl.fr
Sent: 01 February 2024 11:07
To: xios-users@forge.ipsl.jussieu.frmailto:xios-users@forge.ipsl.jussieu.frmailto:xios-users@forge.ipsl.jussieu.frmailto:xios-users@forge.ipsl.jussieu.frmailto:xios-users@forge.ipsl.jussieu.frmailto:xios-users@forge.ipsl.jussieu.frmailto:xios-users@forge.ipsl.jussieu.frmailto:xios-users@forge.ipsl.jussieu.fr
Cc: Mark Hedley
Subject: Re: [XIOS-users] XIOS read into domain

This email was received from an external source. Always check sender details, links & attachments.

Hi Mark,

It seems that you are confusing a domain with the combination of 2 axis, this is not the case here.
You have to define at least the type of the domain (rectilinear, curvilinear, unstructured, gaussian) and coordinates (longitudes, latitudes) used to interpolate smartly the fields.
I let you see the tutorial from slide 31 to 33.

To come back to your xml file, the attribute domain_id does not exist, it is id.
An error should have been generated (in xios_client_0.err) for this kind of typo next to the stack :

In file "attribute_map.cpp", function "xios::CAttribute* xios::CAttributeMap::operator[](const StdString&)", line 126 -> [ key = domain_id] key not found !

(1) **************** void cxios_init_client(const char*, int, int*, int*)

Then you'll see that the description of the domain will be missing :

In file "enum_impl.hpp", function "void xios::CEnum::checkEmpty() const [with T = xios::CDomainAttributes::Enum_type]", line 248 -> Enum is not initialized.

(1) **************** void cxios_context_close_definition()

With a clearer stack :

  File                                    Function                                                                                                  Line

(9) domain.cpp void xios::CDomain::fillInLonLat() 499
(8) nc4_data_input.cpp void xios::CNc4DataInput::readDomainAttributeValueFromFile(xios::CDomain*, std::_cxx11::list<std::p...) 300
(7) nc4_data_input.cpp virtual void xios::CNc4DataInput::readFieldAttributes(xios::CField*, bool) 129
(6) file.cpp void xios::CFile::readAttributesOfEnabledFieldsInReadMode() 731
(5) context.cpp void xios::CContext::readAttributesOfEnabledFieldsInReadModeFiles() 746
(4) context.cpp void xios::CContext::postProcessing() 1549
(3) context.cpp void xios::CContext::postProcessingGlobalAttributes() 580
(2) context.cpp void xios::CContext::closeDefinition() 702
(1) icdata.cpp void cxios_context_close_definition() 118

Regards

Julien

Le 31/01/2024 à 11:06, Mark Hedley a écrit :

Hello XIOS Users

I am trying to form a programme that reads data from input netCDF files, I have a working simple programme that reads 2D data from a file by creating a grid formed of 2 axis instances
The intent here is to explore implementations of resampling operations from input data to output data, using the domain interpolate and similar functionality

I have a working programme using only Axis and Grid,
(see attached resample.F90 & mainAxis.xml (used in iodef.xml))
but I want access to the Domain functionality.

However, when I try to define a Grid based on a Domain from the input file, i get into all sorts of confusion; there are failures, and stack traces i don't understand.
i have tried numerous permutations without success.
(see attached mainDomain.xml (used by changing iodef.xml include) for one failing example, i have tried numerous permutations)

So, I am looking for help in a simple implementation pattern to form a domain from the input dataset on read.
(i have shared the input as CDL, ncgen it to make a netCDf file)

is there some advice or suggestion that you would share with me on this implementation pattern?
all thoughts gratefully received, thank you
mark