Skip to content

Commit beba721

Browse files
committed
DOC: Add section on OpenMP & toc to running_aether.md
I did some digging. Parallelization is weird.
1 parent db2d8fb commit beba721

File tree

1 file changed

+43
-0
lines changed

1 file changed

+43
-0
lines changed

doc/usage/running_aether.md

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,22 @@ This document assumes you have already downloaded and built the Aether model. If
44
not, you should return to [one](../../README.md) of
55
[these](../installation/installation.md) pages before continuing.
66

7+
- [The first run](#the-first-run)
8+
- [Running in 1D](#running-in-1d)
9+
- [Using OpenMP](#using-openmp)
10+
- [Output Files](#output-files)
11+
- [Blocks](#blocks)
12+
- [Ensembles](#ensembles)
13+
- [Post processing](#post-processing)
14+
- [Input Files](#input-files)
15+
- [defaults.json file](#defaultsjson-file)
16+
- [For Developers](#for-developers)
17+
- [aether.json file](#aetherjson-file)
18+
- [planet.in file](#planetin-file)
19+
- [orbits.csv file](#orbitscsv-file)
20+
- [chemistry file](#chemistry-file)
21+
22+
723
## The first run
824

925
Once you have compiled you can run Aether. To remember which runs you're doing,
@@ -70,6 +86,33 @@ to `"sphere"`. No number!
7086
2. Run the code with `./aether`. This will not use MPI, however armadillo may use
7187
multiple OpenMP processes for math, so be careful on cluster login nodes.
7288

89+
### Using OpenMP
90+
91+
> This section is mostly a placeholder. Everything is correct, but has little
92+
> effect on Aether's speed. This is only really a concern on laptops with low core counts
93+
> or shared systems.
94+
95+
Armadillo contains several optimizations which utilize OpenMP for parallelization beyond
96+
the block decomposition on the entire sphere. Thus, runs on 4 MPI processors can benefit
97+
from devoting additional processors to OpenMP parallelization.
98+
99+
The number of OpenMP tasks Armadillo is able to utilize can be set before compiling or
100+
at runtime. To change this *before* compiling, change the value of`ARMA_OPENMP_THREADS`
101+
in the [Armadillo config.hpp file](../../share/include/armadillo_bits/config.hpp#173)
102+
from 8. This will require re-compiling & possibly re-running `cmake`. The more flexible
103+
option is to use a variable at runtime:
104+
105+
The easier way to set the number of OpenMP threads is to use the variable
106+
`OMP_NUM_THREADS`. This can be set before running the executable with
107+
`export OMP_NUM_THREADS=2`, or at runtime with:
108+
109+
```bash
110+
OMP_NUM_THREADS=2 mpirun -np 4 ./aether
111+
```
112+
At this stage in development, there is not much speedup available from OpenMP. For
113+
example, the change in runtime from the default value of 8 (from Armadillo) and 1
114+
(disabling OpenMP) in a 10-minute run is about one minute, or about 10%.
115+
73116
## Output Files
74117
75118
Aether outputs to a subdirectory called UA/output. At this time, all processors

0 commit comments

Comments
 (0)