Building on NERSC Perlmutter

The following was tested with master@eac504c0.

environment setup

Using the default cuda/11.3 module results in a hang during compilation of Omega_h_adj.cpp; cuda/11.1.1 does not have this problem. Note, there is a regression in cuda 11.2 that prevents us from using it to build Omega_h.

module load cmake/3.20.5
module load PrgEnv-gnu
module load cpe-cuda
module swap cuda cuda/11.1.1

build and test

cmake ../omega_h \
  -DCMAKE_INSTALL_PREFIX=$PWD/install \
  -DBUILD_SHARED_LIBS=off \
  -DOmega_h_USE_Kokkos=off \
  -DOmega_h_USE_CUDA=on \
  -DOmega_h_CUDA_ARCH=80 \
  -DOmega_h_USE_MPI=off \
  -DBUILD_TESTING=on \
  -DCMAKE_CXX_COMPILER=CC

make -j8
ctest

Issues

Building with PrgEnv-nvidia/8.2.0 and the default cuda/11.3 results in a hang during the compilation of Omega_h_adj.cpp. Switching the cuda module to cuda/11.1.1 or cuda/11.0.3 results in the following compatability errors:

In file included from /global/homes/c/cwsmith/omegahHackathon/omega_h/src/Omega_h_library.cpp:31:0:
/opt/nvidia/hpc_sdk/Linux_x86_64/21.7/compilers/include/xmmintrin.h:25:2: error: #error "This version of <xmmintrin.h> does not work with this compiler."
 #error "This version of <xmmintrin.h> does not work with this compiler."