Skip to content

Commit 502a636

Browse files
authored
Merge pull request #147 from AetherModel/exchange_messages_interpolation
This allows interpolation on the cubesphere grid ghostcells to have them matchup to the correct locations.
2 parents 0b7071e + 62a93b1 commit 502a636

19 files changed

+1248
-58
lines changed

doc/installation.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -87,7 +87,7 @@ cmake -DCMAKE_CXX_COMPILER=<gcc or mpi executable with full path>
8787
Finally, run the make command in the build directory:
8888

8989
```bash
90-
make -jN
90+
make [-jN]
9191
```
9292

9393
The `-j` flag tells the computer the number of processers to use when running

doc/running_aether.md

Lines changed: 223 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,223 @@
1+
# Running Aether
2+
3+
See the installation.md document for how to install Aether.
4+
5+
## The first run
6+
7+
Once you have compiled you can run Aether. To remember which runs you're doing,
8+
we recommend creating a specific directory for each run. This can be done by
9+
copying the default run directory to have a special name, like this:
10+
```bash
11+
cd ..
12+
cp -R share/run ./run.first_run
13+
```
14+
15+
This creates the directory where you will do your run. In that directory is a link to the aether executable and an input file called aether.json.
16+
17+
You can then run the executable from the directory you created.
18+
```bash
19+
cd run.first_run
20+
./aether
21+
```
22+
You should see something like:
23+
```bash
24+
run.first_run% ./aether
25+
> Need to NOT adjust F10.7, but that isn't included yet!!!
26+
> Writing file : 3DALL_20110320_000000
27+
> Writing file : 3DBFI_20110320_000000
28+
> Wall Time : 4s (left : 1111h); Current Time : 2011 3 20 0 0 0 0
29+
> Wall Time : 4s (left : 23m); Current Time : 2011 3 20 0 0 10 0
30+
> Wall Time : 5s (left : 14m); Current Time : 2011 3 20 0 0 20 0
31+
> Wall Time : 5s (left : 9m); Current Time : 2011 3 20 0 0 30 0
32+
> Wall Time : 5s (left : 7m); Current Time : 2011 3 20 0 0 40 0
33+
> Wall Time : 6s (left : 7m); Current Time : 2011 3 20 0 0 50 0
34+
> Wall Time : 6s (left : 5m); Current Time : 2011 3 20 0 1 0 0
35+
> Wall Time : 6s (left : 5m); Current Time : 2011 3 20 0 1 10 0
36+
> Wall Time : 7s (left : 5m); Current Time : 2011 3 20 0 1 20 0
37+
> Wall Time : 7s (left : 4m); Current Time : 2011 3 20 0 1 30 0
38+
etc.
39+
```
40+
41+
The successful end of this will show a timing summary, similar to:
42+
```bash
43+
>main>advance>Chemistry::calc_chemistry
44+
nTimes called : 720
45+
timing_total (s) : 1.52599
46+
>main>advance>Ions::calc_ion_temperature
47+
nTimes called : 720
48+
timing_total (s) : 19.0279
49+
>main>advance>Neutrals::exchange
50+
nTimes called : 720
51+
timing_total (s) : 2.94398
52+
```
53+
54+
## Output Files
55+
56+
Aether outputs to a subdirectory called UA/output. At this time, all processors within the Aether run output their own files and they can be combined using the post processor.
57+
58+
### Blocks
59+
60+
Aether is a block-based code, so the domain is split up into different blocks that hold a given volume (for spherical grids, this is some range of latitude, longitude and altitude; for other grids it could differ). If you asked for (for example) 4 processors, then the Earth would be broken up into 4 different blocks and each output would consist of 4 different blocks.
61+
62+
### Ensembles
63+
64+
Aether can run the same simulation multiple times simultaneously with different drivers and internal parameters. The collection of these simulations is called an ensemble, with each individual simulation called a member. All of the ensemble members output to the UA/output directory. See the ensembles.md document to learn more about ensembles.
65+
66+
### Post processing
67+
68+
Because there can be multiple blocks and multiple ensemble members, there is a post processor that can pull all of the files together. By default, the post processor combines all of the block files for one ensemble member for one time into one file. This means that if there are 25 hourly outputs requested, there will be 25 total files for each ensemble member.
69+
70+
If there is more than one simulation, then the post processor will create a mean file for each time. If there are more than two ensemble members, a standard deviation file is also created. The mean and standard deviations are calculated across ensemble members.
71+
72+
To run the post processor, do the following:
73+
74+
```bash
75+
cd UA/output
76+
../../../srcPython/post_process.py
77+
```
78+
79+
By default, this will also leave the raw files and will produce plots just to make sure that everything is working ok. If you don't need the raw files (there is not a good reason to keep these, unless you are debugging the post processor), you can run the post-processor with the "-rm" argument. If you don't want any plots, you can run with the "-alt=-1" argument. An example of this is (this is how it is often run):
80+
81+
```bash
82+
../../../srcPython/post_process.py -rm -alt=-1
83+
```
84+
85+
If you are going to be using Aether a lot, you may want to copy the post-processor into your bin directory.
86+
87+
## Input Files
88+
89+
Aether reads in a bunch of files, most of which are specified by the settings file. In order to minimize the number of places where new settings need to be specified, Aether uses a defaults file that sets the generic defaults of the model. This file is in UA/inputs/defaults.json.
90+
91+
## defaults.json file
92+
93+
This is a json file that sets all of the defaults within Aether. This file should never be modified!
94+
95+
### For Developers
96+
97+
Within Aether, the inputs.cpp file has a large handful of of get_ routines to get the values of the settings that the user has set.
98+
99+
## aether.json file
100+
101+
The file aether.json is read in AFTER the defaults file and these settings overwrite the defaults. So, if you want to modify the default settings, simply copy a setting out of defaults.json and paste it into aether.json. Then, you can modify the setting in aether.json.
102+
103+
Because the files are json files, you don't actually have to set all of the subsettings within a specific setting. For example, within the defaults.json file, the EUV setting is:
104+
105+
```bash
106+
"Euv" : {
107+
"doUse" : true,
108+
"Model" : "euvac",
109+
"File" : "UA/inputs/euv.csv",
110+
"IncludePhotoElectrons" : true,
111+
"HeatingEfficiency" : 0.05,
112+
"dt" : 60.0},
113+
```
114+
115+
If you simply want to turn off the EUV, you can insert this into the aether.json file:
116+
117+
```bash
118+
"Euv" : {
119+
"doUse" : false},
120+
```
121+
122+
or, if you wanted to use the 59 wavelength bins instead of the 37 EUVAC bins, you could do:
123+
124+
```bash
125+
"Euv" : {
126+
"File" : "UA/inputs/euv_59.csv"},
127+
```
128+
129+
## planet.in file
130+
131+
Within the input file, the planet file can be set:
132+
133+
```bash
134+
"Planet" : {
135+
"name" : "earth",
136+
"file": "UA/inputs/earth.in"},
137+
```
138+
139+
This file defines the neutrals and ions to be modeled. For example:
140+
141+
```bash
142+
#NEUTRALS
143+
name, mass, vibration, thermal_cond, thermal_exp, advect, BC
144+
O, 16, 5, 5.6e-4, 0.69, 1, 5.0e17
145+
N, 14, 5, 5.6e-4, 0.69, 1, 2.5e11
146+
O2, 32, 7, 3.6e-4, 0.69, 1, 4.0e18
147+
N2, 28, 7, 3.6e-4, 0.69, 1, 1.7e19
148+
NO, 30, 7, 3.6e-4, 0.69, 1, 5.4e14
149+
He, 4, 5, 5.6e-4, 0.69, 1, 1.5e14
150+
N_2D, 14, 5, 5.6e-4, 0.69, 0, 2.5e9
151+
N_2P, 14, 5, 5.6e-4, 0.69, 0, 2.5e7
152+
H, 1, 5, 5.6e-4, 0.69, 0, 3.0e13
153+
O_1D, 16, 5, 5.6e-4, 0.69, 0, 1.0e10
154+
CO2, 46, 7, 3.6e-4, 0.69, 0, 4.5e15
155+
```
156+
157+
This is a comma seperated list that includes:
158+
- name - this is the common name of the species
159+
- mass - the atomic mass of the species
160+
- vibration - the degrees of freedom for the species
161+
- thermal_cond - the thermal conductivity coeficient of the species
162+
- thermal_exp - the exponent that is put on the temperature (Lambda = A * T^B)
163+
- advect - whether the species is advected or not
164+
- BC - the lower boundary condition on the species density
165+
166+
```bash
167+
#IONS
168+
name, mass, charge, advect
169+
O+, 16, 1, 1
170+
O2+, 32, 1, 0
171+
N2+, 28, 1, 0
172+
NO+, 30, 1, 0
173+
N+, 14, 1, 0
174+
He+, 4, 1, 1
175+
O+_2D, 16, 1, 0
176+
O+_2P, 16, 1, 0
177+
```
178+
179+
This is a comma separated list that includes:
180+
- name - this is the common name of the species
181+
- mass - the atomic mass of the species
182+
- charge - the charge of the species
183+
- advect - whether the species is advected or not
184+
185+
```bash
186+
#TEMPERATURE
187+
alt, temp
188+
100.0, 200.0
189+
200.0, 800.0
190+
300.0, 800.0
191+
```
192+
193+
Within the input file (i.e., aether.json), the following can be set:
194+
195+
```bash
196+
"InitialConditions" : {
197+
"type" : "Planet"},
198+
199+
"BoundaryConditions" : {
200+
"type" : "Planet"},
201+
```
202+
203+
If these are set to "Planet", then the temperature profile is set as the initial condition. If the boundary condition is set to "Planet", then the lowest altitude is used as a boundary condition on the temperature.
204+
205+
## orbits.csv file
206+
207+
This file contains all of the orbital, mass, rotation, and magnetic field characteristics of the planets. All of the planets within the solar system are included. Other planets (like exo-planets or artificial planets) can be included by adding lines.
208+
209+
## chemistry file
210+
211+
The chemistry file defines all of the chemical reactions within the system.
212+
213+
For example:
214+
```bash
215+
He+ + e- -> He, with R = 4.8e-18 * (250 / Te) ^ 0.7
216+
```
217+
With an exothermal energy of xxx. Also, the uncertainty of the reaction can be set too (set to 10%, or 0.1).
218+
219+
```bash
220+
R11,He+,e-,,,He,,,4.80E-18,(250/Te)^0.7,,1,0,,0.1,,,,gitm,250,Te,0.7,,,,1,
221+
```
222+
223+

doc/student.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,9 @@ emacs can be installed using a package manager such as apt or port
2626

2727
21st Century Editors:
2828

29-
1. There are always popular Integrated Development Environments (IDEs) that can
29+
1. Most people are using Visual Studio Code to work on Aether.
30+
31+
2. There are always popular Integrated Development Environments (IDEs) that can
3032
be used to edit, develop, and test code. If you use one and develop on
3133
Aether, please feel free to update this section.
3234

@@ -44,7 +46,7 @@ this:
4446
3. Click "Create fork"
4547

4648
4. GitHub will take you to this repository automatically. It is now your
47-
"version" of Aether, and all development will be done here. Will will use
49+
"version" of Aether, and all development will be done here. We will use
4850
the username "YourRepo" instead of your actual repository in the
4951
examples below.
5052

include/grid.h

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ class Grid {
3030
arma_cube A11_inv_scgc, A12_inv_scgc, A21_inv_scgc, A22_inv_scgc;
3131
arma_cube g11_upper_scgc, g12_upper_scgc, g21_upper_scgc, g22_upper_scgc;
3232
arma_cube sqrt_g_scgc;
33+
arma_cube refx_angle, refy_angle;
3334

3435
// Edge/Corner coordinates
3536
arma_cube geoLon_Left;
@@ -52,6 +53,8 @@ class Grid {
5253
arma_cube A11_inv_Left, A12_inv_Left, A21_inv_Left, A22_inv_Left;
5354
arma_cube g11_upper_Left, g12_upper_Left, g21_upper_Left, g22_upper_Left;
5455
arma_cube sqrt_g_Left;
56+
arma_mat refx_angle_Left, refy_angle_Left;
57+
arma_mat refx_angle_Down, refy_angle_Down;
5558

5659
arma_cube A11_Down, A12_Down, A21_Down, A22_Down;
5760
arma_cube A11_inv_Down, A12_inv_Down, A21_inv_Down, A22_inv_Down;
@@ -215,9 +218,16 @@ class Grid {
215218
MPI_Request requests;
216219
precision_t* buffer;
217220
precision_t* rbuffer;
221+
222+
// For cubesphere. these are needed for interpolation
223+
// when the cells go onto a different face:
224+
arma_mat index;
225+
arma_mat ratio;
218226
};
219227

220228
std::vector<messages_struct> interchanges;
229+
std::vector<messages_struct> interchangesOneVar;
230+
bool gcInterpolationSet = false;
221231

222232
messages_struct make_new_interconnection(int64_t iDir,
223233
int64_t nVars,
@@ -229,7 +239,9 @@ class Grid {
229239
bool XbecomesY);
230240

231241
bool send_one_face(int64_t iFace);
242+
bool send_one_var_one_face(int64_t iFace);
232243
bool receive_one_face(int64_t iFace);
244+
bool receive_one_var_one_face(int64_t iFace);
233245

234246
/**
235247
* \brief Set the interpolation coefficients

include/parallel.h

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -79,4 +79,45 @@ bool unpack_border(arma_cube &value,
7979
bool DoReverseY,
8080
bool XbecomesY);
8181

82+
/**********************************************************************
83+
\brief initialize the grid variables to set up ghostcell message passing
84+
\param grid the grid to set up message passing on
85+
\param nVarsToPass how many variables to pass
86+
**/
87+
88+
bool exchange_sides_init(Grid &grid, int64_t nVarsToPass);
89+
90+
/**********************************************************************
91+
\brief exchange one variable's ghost cells to adjacent blocks
92+
\param grid the grid that describes the system
93+
\param vat_to_pass is variable to pass
94+
\param doReverseSignAcrossPole true for east/north vector components
95+
**/
96+
97+
bool exchange_one_var(Grid &grid,
98+
arma_cube &var_to_pass,
99+
bool doReverseSignAcrossPole);
100+
101+
/**********************************************************************
102+
\brief test the exchange messages one var function
103+
\param grid the grid that describes the system
104+
**/
105+
106+
// -----------------------------------------------------------------------------
107+
// -----------------------------------------------------------------------------
108+
109+
arma_cube interpolate_ghostcells(arma_cube varIn, Grid &grid);
110+
111+
// -----------------------------------------------------------------------------
112+
// -----------------------------------------------------------------------------
113+
114+
bool test_ghostcell_interpolation(Grid &grid);
115+
116+
// -----------------------------------------------------------------------------
117+
// -----------------------------------------------------------------------------
118+
119+
bool find_ghostcell_interpolation_coefs(Grid &grid);
120+
121+
122+
82123
#endif // INCLUDE_PARALLEL_H_

include/times.h

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -94,6 +94,13 @@ class Times {
9494
**/
9595
std::string get_YMD_HMS();
9696

97+
/**************************************************************
98+
\brief Returns the current time as a string (year, month...)
99+
100+
Returns the current time as a string of format YYYYMMDD_HHMM00
101+
**/
102+
std::string get_YMD_HM0();
103+
97104
/**************************************************************
98105
\brief Returns the intermediate time in seconds since ref date
99106
**/
@@ -221,6 +228,9 @@ class Times {
221228
/// represented as YYYYMMDD_HHMMSS
222229
std::string sYMD_HMS;
223230

231+
/// represented as YYYYMMDD_HHMM00
232+
std::string sYMD_HM0;
233+
224234
// -------------------------------------------------------------
225235
// Keeping track of walltime for the run:
226236

0 commit comments

Comments
 (0)