ZEN-30304/ZEN-31048: PKG/SPEC - expanse/0.17.3/gpu/b - User requested updated version of LAMMPS

[mkandes]

Original request can be found here: https://sdsc.zendesk.com/agent/tickets/30304

After deploying an initial spec into production for testing, another user found that OpenMP support is not included. This ticket can be found here: https://sdsc.zendesk.com/agent/tickets/31048

I was able to confirm the issue with OpenMP for both recently installed versions in expanse/0.17.3/gpu/b.

LAMMPS (10 Mar 2021)
  using 10 OpenMP thread(s) per MPI task
ERROR: Using suffix omp without USER-OMP package installed (src/src/lammps.cpp:843)
Last command: (unknown)
real 1.10
user 0.11
sys 0.16

LAMMPS (2 Aug 2023)
  using 10 OpenMP thread(s) per MPI task
ERROR: Package omp command without OPENMP package installed (src/src/input.cpp:1700)
Last command: package omp 10
real 0.88
user 0.07
sys 0.14

It seems like maybe both the +openmp and +user-openmp variants need to be enabled for OpenMP to be enabled. These are effectively the same codebase, so it's unclear why the Spack package does not properly account for this in its implementation. Something that may need to be fixed. For reference, here are the starting specs for both builds.

[mkandes@login02 ~]$ spack find -lvd lammps
==> 3 installed packages
-- linux-rocky8-cascadelake / gcc@10.2.0 ------------------------
wgrzixt lammps@20230802+asphere+body+class2+colloid+compress+coreshell+cuda~cuda_mps+dipole~exceptions~ffmpeg+granular~ipo+jpeg+kim+kokkos+kspace~latte+lib+manybody+mc~meam+misc+mliap+molecule+mpi+mpiio~opencl+openmp+opt+peri+png+poems+python+qeq+replica+rigid+shock+snap+spin+srd~user-adios+user-atc+user-awpmd+user-bocs+user-cgsdk+user-colvars+user-diffraction+user-dpd+user-drude+user-eff+user-fep~user-h5md+user-lb+user-manifold+user-meamc+user-mesodpd+user-mesont+user-mgpt+user-misc+user-mofff~user-netcdf~user-omp+user-phonon~user-plumed+user-ptm+user-qtb+user-reaction+user-reaxc+user-sdpd+user-smd+user-smtbq+user-sph+user-tally+user-uef+user-yaff+voronoi build_type=RelWithDebInfo cuda_arch=70
blza2ps     cuda@11.2.2~dev
2q4yola         libxml2@2.9.12~python
5a3xt3s             libiconv@1.16 libs=shared,static
5xho2dj             xz@5.2.5~pic libs=shared,static
2c5fvip             zlib@1.2.11+optimize+pic+shared
aoih524     eigen@3.4.0~ipo build_type=RelWithDebInfo
7ahyh5v     fftw@3.3.10~mpi~openmp~pfft_patches precision=double,float
eqgx2bp     kim-api@2.2.1~ipo build_type=RelWithDebInfo
phogmfw     kokkos@3.7.02~aggressive_vectorization~compiler_warnings+cuda+cuda_constexpr+cuda_lambda+cuda_ldg_intrinsic~cuda_relocatable_device_code~cuda_uvm~debug~debug_bounds_check~debug_dualview_modify_check~deprecated_code~examples~explicit_instantiation~hpx~hpx_async_dispatch~hwloc~ipo~memkind~numactl~openmp+pic+profiling~profiling_load_print~pthread~qthread~rocm+serial+shared~sycl~tests~tuning+wrapper amdgpu_target=none build_type=RelWithDebInfo cuda_arch=70 std=14
7urw4af         kokkos-nvcc-wrapper@3.2.00+mpi
gzzscfu             openmpi@4.1.3~atomics+cuda~cxx~cxx_exceptions~gpfs~internal-hwloc~java+legacylaunchers+lustre~memchecker+pmi+pmix+romio~rsh~singularity+static+vt+wrapper-rpath cuda_arch=70,80 fabrics=ucx schedulers=slurm
okiyq35                 hwloc@2.6.0~cairo+cuda~gl~libudev+libxml2~netloc~nvml~opencl+pci~rocm+shared
x3u2rw4                     libpciaccess@0.16
5jrknc3                     ncurses@6.2~symlinks+termlib abi=none
ne2joyw                 libevent@2.1.8~openssl
73aggpy                 lustre@2.15.2
34rinp4                 numactl@2.0.14 patches=4e1d78cbbb85de625bad28705e748856033eaafab92a66dffd383a3d7e00cc94,62fc8a8bf7665a60e8f4c93ebbd535647cebf74198f7afafec4c085a8825c006,ff37630df599cfabf0740518b91ec8daaf18e8f288b19adaae5364dc1f6b2296
43uenl2                 pmix@3.2.1~docs+pmi_backwards_compatibility~restful
nflzb3l                 slurm@21.08.8~gtk~hdf5~hwloc~mariadb~pmix+readline~restd sysconfdir=PREFIX/etc
msro2p7                 ucx@1.10.1~assertions~cm+cma+cuda+dc~debug+dm+gdrcopy+ib-hw-tm~java~knem~logging+mlx5-dv+optimizations~parameter_checking+pic+rc~rocm+thread_multiple+ud~xpmem cuda_arch=70,80
nefrbxp                     gdrcopy@2.2
k7hbyv2                     rdma-core@43.0~ipo build_type=RelWithDebInfo
uwaulm2     libjpeg-turbo@2.1.0
a7krc3b     libpng@1.6.37
lsmegf6     openblas@0.3.18~bignuma~consistent_fpcsr~ilp64+locking+pic+shared threads=none
uasyy5n     python@3.8.12+bz2+ctypes+dbm~debug+libxml2+lzma~nis+optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl~tix~tkinter~ucs4+uuid+zlib patches=0d98e93189bc278fbc37a50ed7f183bd8aaf249a8e1670a465f0db6bb4f8cf87,4c2457325f2b608b1b6a2c63087df8c26e07db3e3d493caf36a56f0ecf6fb768,f2fd060afc4b4618fe8104c4c5d771f36dc55b1db5a4623785a4ea707ec72fb4
nlagguk         bzip2@1.0.8~debug~pic+shared
5hqjubn         expat@2.4.1+libbsd
c6r2tgf             libbsd@0.11.3
v3ceevj                 libmd@1.0.3
zlcmcv5         gdbm@1.19
57jahak             readline@8.1
4mzhvmy         gettext@0.21+bzip2+curses+git~libunistring+libxml2+tar+xz
vlxqcge             tar@1.34
zvema6a         libffi@3.3 patches=26f26c6f29a7ce9bf370ad3ab2610f99365b4bdd7b82e7c31df41a3370d685c0
47xsvyk         openssl@1.1.1k~docs certs=system
uqy7ybg         sqlite@3.36.0+column_metadata+fts~functions~rtree
fjihfsz         util-linux-uuid@2.36.2
w3k24sh     voropp@0.4.6+pic


-- linux-rocky8-skylake_avx512 / gcc@8.4.0 ----------------------
ytjmvfx lammps@20210310+asphere+body+class2+colloid+compress+coreshell+cuda~cuda_mps+dipole~exceptions~ffmpeg+granular~ipo+jpeg+kim+kokkos+kspace~latte+lib+manybody+mc~meam+misc+mliap+molecule+mpi+mpiio~opencl+openmp+opt+peri+png+poems+python+qeq+replica+rigid+shock+snap+spin+srd~user-adios+user-atc+user-awpmd+user-bocs+user-cgsdk+user-colvars+user-diffraction+user-dpd+user-drude+user-eff+user-fep~user-h5md+user-lb+user-manifold+user-meamc+user-mesodpd+user-mesont+user-mgpt+user-misc+user-mofff~user-netcdf~user-omp+user-phonon+user-plumed+user-ptm+user-qtb+user-reaction+user-reaxc+user-sdpd+user-smd+user-smtbq+user-sph+user-tally+user-uef+user-yaff+voronoi build_type=RelWithDebInfo cuda_arch=70
yx5cxnu     cuda@10.2.89~dev
cctghrh         libxml2@2.9.12~python
ksynfmj             libiconv@1.16 libs=shared,static
udmvado             xz@5.2.5~pic libs=shared,static
7apv7tj             zlib@1.2.11+optimize+pic+shared
ygkbggs     eigen@3.4.0~ipo build_type=RelWithDebInfo
qfar473     fftw@3.3.10~mpi~openmp~pfft_patches precision=double,float
iiwtfrq     kim-api@2.2.1~ipo build_type=RelWithDebInfo
jh4pw54     kokkos@3.4.01~aggressive_vectorization~compiler_warnings+cuda+cuda_constexpr+cuda_lambda+cuda_ldg_intrinsic~cuda_relocatable_device_code~cuda_uvm~debug~debug_bounds_check~debug_dualview_modify_check~deprecated_code~examples~explicit_instantiation~hpx~hpx_async_dispatch~hwloc~ipo~memkind~numactl~openmp+pic+profiling~profiling_load_print~pthread~qthread~rocm+serial+shared~sycl~tests~tuning+wrapper amdgpu_target=none build_type=RelWithDebInfo cuda_arch=70 std=14
hxjlokx         kokkos-nvcc-wrapper@3.2.00+mpi
v2ei3ge             openmpi@4.1.3~atomics+cuda~cxx~cxx_exceptions~gpfs~internal-hwloc~java+legacylaunchers+lustre~memchecker+pmi+pmix+romio~rsh~singularity+static+vt+wrapper-rpath cuda_arch=70 fabrics=ucx schedulers=slurm
7evhgqy                 hwloc@2.6.0~cairo+cuda~gl~libudev+libxml2~netloc~nvml~opencl+pci~rocm+shared
dusortv                     libpciaccess@0.16
fj4m6bg                     ncurses@6.2~symlinks+termlib abi=none
wgoayfz                 libevent@2.1.8+openssl
um4do2r                     openssl@1.1.1k~docs certs=system
dfynryt                 lustre@2.15.2
gpjef7m                 numactl@2.0.14 patches=4e1d78cbbb85de625bad28705e748856033eaafab92a66dffd383a3d7e00cc94,62fc8a8bf7665a60e8f4c93ebbd535647cebf74198f7afafec4c085a8825c006,ff37630df599cfabf0740518b91ec8daaf18e8f288b19adaae5364dc1f6b2296
gl4mdry                 pmix@3.2.1~docs+pmi_backwards_compatibility~restful
mctz53r                 slurm@21.08.8~gtk~hdf5~hwloc~mariadb~pmix+readline~restd sysconfdir=PREFIX/etc
fkguskk                 ucx@1.10.1~assertions~cm+cma+cuda+dc~debug+dm+gdrcopy+ib-hw-tm~java~knem~logging+mlx5-dv+optimizations~parameter_checking+pic+rc~rocm+thread_multiple+ud~xpmem cuda_arch=70
ntvmzwk                     gdrcopy@2.2
53wlz3i                     rdma-core@43.0~ipo build_type=RelWithDebInfo
zne4hoj     libjpeg-turbo@2.1.0
3trf423     libpng@1.6.37
n4ay3co     openblas@0.3.18~bignuma~consistent_fpcsr~ilp64+locking+pic+shared threads=none
5u64glq     plumed@2.6.3+gsl+mpi+shared arrayfire=none optional_modules=all
gfpesxa         gsl@2.7~external-cblas
nmsf2rf     python@3.8.12+bz2+ctypes+dbm~debug+libxml2+lzma~nis~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl~tix~tkinter~ucs4+uuid+zlib patches=0d98e93189bc278fbc37a50ed7f183bd8aaf249a8e1670a465f0db6bb4f8cf87,4c2457325f2b608b1b6a2c63087df8c26e07db3e3d493caf36a56f0ecf6fb768,f2fd060afc4b4618fe8104c4c5d771f36dc55b1db5a4623785a4ea707ec72fb4
zg6j6o4         bzip2@1.0.8~debug~pic+shared
2pgh5u7         expat@2.4.1+libbsd
efhliq2             libbsd@0.11.3
wvibiuk                 libmd@1.0.3
glugi6a         gdbm@1.19
6wqnjhx             readline@8.1
iap64mm         gettext@0.21+bzip2+curses+git~libunistring+libxml2+tar+xz
p7jtkdm             tar@1.34
w6bcjqa         libffi@3.3 patches=26f26c6f29a7ce9bf370ad3ab2610f99365b4bdd7b82e7c31df41a3370d685c0
mezd2bx         sqlite@3.36.0+column_metadata+fts~functions~rtree
fajrrs3         util-linux-uuid@2.36.2
4pcy3u7     voropp@0.4.6+pic

I will attempt to enable OpenMP by rebuilding lammps@20230802 with +user-omp enabled, keeping +openmp on as well.