Allow remapper to take multiple Fields

This PR increases the rank of some internal arrays of the remapper by 1. The new
dimension is to allow the remapper to process multiple Fields at the same time.

The idea is the following: when creating a Remapper, one can specify a
buffer_length. When the buffer_length is larger than one, the Remapper will
preallocate space to be able to interpolate buffer_length at the same time.
Then, when `interpolate` is called, the Remapper can work with any number of
Fields and the work is divided in batches of buffer_length size.

E.g, If buffer_length is 10 and 22 fields are to be interpolated, the work is
processed in groups of 10+10+2. The only cost of choosing a large buffer_length
is memory (there shouldn't be any runtime penalty in interpolating 1 field with
a batch_length of 100).

The memory cost of a Remapper is order (B, H, V), where B is the buffer length,
H is the number of horizontal points, and V the number of vertical points. For H
= 180x90 and V = 50, this means that each buffer costs 51_840_000 bytes (50 MB)
for double precision on the root process + 50 MB / N_tasks on each task + base
cost that is independent of B.

Author: Sbozzolo

.buildkite/pipeline.yml

@@ -243,30 +243,38 @@ steps:
         agents:
           slurm_ntasks: 4
 
-      - label: "Unit: distributed remapping"
-        key: distributed_remapping
+      - label: "Unit: distributed remapping (1 process)"
+        key: distributed_remapping_1proc
+        command: "julia --color=yes --check-bounds=yes --project=test test/Remapping/distributed_remapping.jl"
+        env:
+          CLIMACOMMS_DEVICE: "CPU"
+
+      - label: "Unit: distributed remapping (2 processes)"
+        key: distributed_remapping_2procs
         command: "srun julia --color=yes --check-bounds=yes --project=test test/Remapping/distributed_remapping.jl"
         env:
           CLIMACOMMS_CONTEXT: "MPI"
           CLIMACOMMS_DEVICE: "CPU"
         agents:
           slurm_ntasks: 2
 
-      - label: "Unit: distributed remapping (1 process)"
-        key: distributed_remapping_1proc
+      - label: "Unit: distributed remapping with CUDA (1 process)"
+        key: distributed_remapping_gpu_1proc
         command: "julia --color=yes --check-bounds=yes --project=test test/Remapping/distributed_remapping.jl"
         env:
-          CLIMACOMMS_DEVICE: "CPU"
+          CLIMACOMMS_DEVICE: "CUDA"
+        agents:
+          slurm_gpus: 1
 
-      - label: "Unit: distributed remapping with CUDA"
-        key: distributed_remapping_gpu
+      - label: "Unit: distributed remapping with CUDA (2 processes)"
+        key: distributed_remapping_gpu_2procs
         command: "srun julia --color=yes --check-bounds=yes --project=test test/Remapping/distributed_remapping.jl"
         env:
           CLIMACOMMS_CONTEXT: "MPI"
           CLIMACOMMS_DEVICE: "CUDA"
         agents:
           slurm_ntasks: 2
-          slurm_gpus: 1
+          slurm_gpus_per_task: 1
 
       - label: "Unit: distributed gather"
         key: unit_distributed_gather4

NEWS.md

@@ -4,6 +4,9 @@ ClimaCore.jl Release Notes
 main
 -------
 
+- ![][badge-🤖precisionΔ] `Remapper`s can now process multiple `Field`s at the same time if created with some `buffer_lenght > 1`.
+  PR ([#1669](https://github.com/CliMA/ClimaCore.jl/pull/1669)) Machine-precision differences are expected.
+
 v0.13.4
 -------