Is it safe to save copies of boxarray/distromap in fortran make_level_from_scratch "virtual" functions

[pbrady]

We're using the Fortran interface with registered "virtual" functions via amrex_init_virtual_functions. Our mk_lev_scratch function eventually saves the passed in boxarray and distromap arguments:

    ...
    type(amrex_boxarray), intent(in) :: ba
    type(amrex_distromap), intent(in) :: dm
    ... 
    !! Set the mesh data for this level
    this%ba(level) = ba
    this%dm(level) = dm

The initialization runs successfully and creates the expected number of levels (6 in our test case). However, we get a segfault when using these saved this%ba(level) and this%dm(level) amrex datatypes when trying to build multifabs after the initialization has been completed. I find that replacing our use of this%ba(level) with calls to amrex_get_boxarray(level) (and similarly for this%dm) fixes the segfault.

Are we not allowed to save instances of the boxarray and distromap that are passed into the amr callbacks?

[WeiqunZhang]

Right, it is not safe to use BoxArray and DistributionMapping saved like that. The technical detail is that the DistributionMapping passed to the Fortran function is a local variable in C++. It's gone after the call. There are three ways if you want to save BoxArray and DistributionMapping.

1. Do "deep" copy. It's deep in the sense that it will increment a reference counter in C++, not deep-copying all the data.

    call amrex_boxarray_clone(this%ba(level), ba)
    call amrex_disromap_clone(this%dm(level), dm)

2. Create a shallow copy from a Fortran multifab. If the mulitfab is gone later, the shallow copy will be invalid.

    this%ba(level) = mf%ba
    this%dm(level) = mf%dm

3. As you have figured it out, call amrex_get_boxarray(level) and amrex_get_distromap(level).

amrex_get_boxarray and amrex_get_distromap are cheap. So you don't really need to save them. You can call them whenever you need BoxArray and DistributionMapping.