Skip to content

use Arm GitHub Actions runner when checking for missing installations for aarch64/* CPU targets #1046

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 32 additions & 21 deletions .github/workflows/test-software.eessi.io.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,30 +8,41 @@ on:
permissions:
contents: read # to fetch code (actions/checkout)
env:
EESSI_VERSION: '2023.06'
EESSI_ACCELERATOR_TARGETS: |
x86_64/amd/zen2:
- nvidia/cc80
x86_64/amd/zen3:
- nvidia/cc80
jobs:
check_missing:
runs-on: ubuntu-24.04
strategy:
fail-fast: false
matrix:
EESSI_VERSION:
- 2023.06
EESSI_SOFTWARE_SUBDIR_OVERRIDE:
- aarch64/generic
- aarch64/neoverse_n1
- aarch64/neoverse_v1
- x86_64/amd/zen2
- x86_64/amd/zen3
- x86_64/amd/zen4
- x86_64/intel/haswell
- x86_64/intel/sapphirerapids
- x86_64/intel/skylake_avx512
- x86_64/generic
include:
# Arm CPU targets
- runs_on: ubuntu-24.04-arm
EESSI_SOFTWARE_SUBDIR_OVERRIDE: aarch64/generic
- runs_on: ubuntu-24.04-arm
EESSI_SOFTWARE_SUBDIR_OVERRIDE: aarch64/neoverse_n1
- runs_on: ubuntu-24.04-arm
EESSI_SOFTWARE_SUBDIR_OVERRIDE: aarch64/neoverse_v1
# x86_64 CPU targets
- runs_on: ubuntu-24.04
EESSI_SOFTWARE_SUBDIR_OVERRIDE: x86_64/amd/zen2
- runs_on: ubuntu-24.04
EESSI_SOFTWARE_SUBDIR_OVERRIDE: x86_64/amd/zen3
- runs_on: ubuntu-24.04
EESSI_SOFTWARE_SUBDIR_OVERRIDE: x86_64/amd/zen4
- runs_on: ubuntu-24.04
EESSI_SOFTWARE_SUBDIR_OVERRIDE: x86_64/intel/haswell
- runs_on: ubuntu-24.04
EESSI_SOFTWARE_SUBDIR_OVERRIDE: x86_64/intel/sapphirerapids
- runs_on: ubuntu-24.04
EESSI_SOFTWARE_SUBDIR_OVERRIDE: x86_64/intel/skylake_avx512
- runs_on: ubuntu-24.04
EESSI_SOFTWARE_SUBDIR_OVERRIDE: x86_64/generic
runs-on: ${{ matrix.runs_on }}
steps:
- name: Check out software-layer repository
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
Expand All @@ -46,20 +57,20 @@ jobs:
- name: Test check_missing_installations.sh script
run: |
export EESSI_SOFTWARE_SUBDIR_OVERRIDE=${{matrix.EESSI_SOFTWARE_SUBDIR_OVERRIDE}}
source /cvmfs/software.eessi.io/versions/${{matrix.EESSI_VERSION}}/init/bash
source /cvmfs/software.eessi.io/versions/${EESSI_VERSION}/init/bash
# set $EESSI_CPU_FAMILY to the CPU architecture that corresponds to $EESSI_SOFTWARE_SUBDIR_OVERRIDE (part before the first slash),
# to prevent issues with checks in the Easybuild configuration that use this variable
export EESSI_CPU_FAMILY=${EESSI_SOFTWARE_SUBDIR_OVERRIDE%%/*}
module load EasyBuild
which eb
eb --version
export EESSI_PREFIX=/cvmfs/software.eessi.io/versions/${{matrix.EESSI_VERSION}}
export EESSI_PREFIX=/cvmfs/software.eessi.io/versions/${EESSI_VERSION}
export EESSI_OS_TYPE=linux
env | grep ^EESSI | sort

# first check the CPU-only builds for this CPU target
echo "just run check_missing_installations.sh (should use easystacks/software.eessi.io/${{matrix.EESSI_VERSION}}/eessi-${{matrix.EESSI_VERSION}}-*.yml with latest EasyBuild release)"
for easystack_file in $(EESSI_VERSION=${{matrix.EESSI_VERSION}} .github/workflows/scripts/only_latest_easystacks.sh); do
echo "just run check_missing_installations.sh (should use easystacks/software.eessi.io/${EESSI_VERSION}/eessi-${EESSI_VERSION}-*.yml with latest EasyBuild release)"
for easystack_file in $(EESSI_VERSION=${EESSI_VERSION} .github/workflows/scripts/only_latest_easystacks.sh); do
echo "check missing installations for ${easystack_file}..."
./check_missing_installations.sh ${easystack_file}
ec=$?
Expand All @@ -74,7 +85,7 @@ jobs:
for accel in ${accelerators}; do
module use ${EESSI_SOFTWARE_PATH}/accel/${accel}/modules/all
echo "checking missing installations for accelerator ${accel} using modulepath: ${MODULEPATH}"
for easystack_file in $(EESSI_VERSION=${{matrix.EESSI_VERSION}} ACCEL_EASYSTACKS=1 .github/workflows/scripts/only_latest_easystacks.sh); do
for easystack_file in $(EESSI_VERSION=${EESSI_VERSION} ACCEL_EASYSTACKS=1 .github/workflows/scripts/only_latest_easystacks.sh); do
echo "check missing installations for ${easystack_file}..."
./check_missing_installations.sh ${easystack_file}
ec=$?
Expand All @@ -87,14 +98,14 @@ jobs:
- name: Test check_missing_installations.sh with missing package (GCC/8.3.0)
run: |
export EESSI_SOFTWARE_SUBDIR_OVERRIDE=${{matrix.EESSI_SOFTWARE_SUBDIR_OVERRIDE}}
source /cvmfs/software.eessi.io/versions/${{matrix.EESSI_VERSION}}/init/bash
source /cvmfs/software.eessi.io/versions/${EESSI_VERSION}/init/bash
# set $EESSI_CPU_FAMILY to the CPU architecture that corresponds to $EESSI_SOFTWARE_SUBDIR_OVERRIDE (part before the first slash),
# to prevent issues with checks in the Easybuild configuration that use this variable
export EESSI_CPU_FAMILY=${EESSI_SOFTWARE_SUBDIR_OVERRIDE%%/*}
module load EasyBuild
which eb
eb --version
export EESSI_PREFIX=/cvmfs/software.eessi.io/versions/${{matrix.EESSI_VERSION}}
export EESSI_PREFIX=/cvmfs/software.eessi.io/versions/${EESSI_VERSION}
export EESSI_OS_TYPE=linux
env | grep ^EESSI | sort
# create dummy easystack file with a single entry (something that is not installed in EESSI)
Expand Down
36 changes: 0 additions & 36 deletions eb_hooks.py
Original file line number Diff line number Diff line change
Expand Up @@ -373,40 +373,6 @@ def parse_hook_freeimage_aarch64(ec, *args, **kwargs):
print_msg("Changed toolchainopts for %s: %s", ec.name, ec['toolchainopts'])


def parse_hook_lammps_remove_deps_for_aarch64(ec, *args, **kwargs):
"""
Remove x86_64 specific dependencies for the CI and missing installations to pass on aarch64
"""
if ec.name == 'LAMMPS':
if ec.version in ('2Aug2023_update2', '29Aug2024'):
if os.getenv('EESSI_CPU_FAMILY') == 'aarch64':
# ScaFaCoS and tbb are not compatible with aarch64/* CPU targets,
# so remove them as dependencies for LAMMPS (they're optional);
# see also https://github.com/easybuilders/easybuild-easyconfigs/pull/19164 +
# https://github.com/easybuilders/easybuild-easyconfigs/pull/19000;
# we need this hook because we check for missing installations for all CPU targets
# on an x86_64 VM in GitHub Actions (so condition based on ARCH in LAMMPS easyconfig is always true)
ec['dependencies'] = [dep for dep in ec['dependencies'] if dep[0] not in ('ScaFaCoS', 'tbb',)]
else:
raise EasyBuildError("LAMMPS-specific hook triggered for non-LAMMPS easyconfig?!")


def parse_hook_CP2K_remove_deps_for_aarch64(ec, *args, **kwargs):
"""
Remove x86_64 specific dependencies for the CI and missing installations to pass on aarch64
"""
if ec.name == 'CP2K' and ec.version in ('2023.1',):
if os.getenv('EESSI_CPU_FAMILY') == 'aarch64':
# LIBXSMM is not supported on ARM with GCC 12.2.0 and 12.3.0
# See https://www.cp2k.org/dev:compiler_support
# See https://github.com/easybuilders/easybuild-easyconfigs/pull/20951
# we need this hook because we check for missing installations for all CPU targets
# on an x86_64 VM in GitHub Actions (so condition based on ARCH in LAMMPS easyconfig is always true)
ec['dependencies'] = [dep for dep in ec['dependencies'] if dep[0] not in ('libxsmm',)]
else:
raise EasyBuildError("CP2K-specific hook triggered for non-CP2K easyconfig?!")


def parse_hook_zen4_module_only(ec, eprefix):
"""
Use --force --module-only if building a foss-2022b-based EasyConfig for Zen4.
Expand Down Expand Up @@ -1192,8 +1158,6 @@ def post_module_hook(self, *args, **kwargs):
'fontconfig': parse_hook_fontconfig_add_fonts,
'FreeImage': parse_hook_freeimage_aarch64,
'grpcio': parse_hook_grpcio_zlib,
'LAMMPS': parse_hook_lammps_remove_deps_for_aarch64,
'CP2K': parse_hook_CP2K_remove_deps_for_aarch64,
'OpenBLAS': parse_hook_openblas_relax_lapack_tests_num_errors,
'pybind11': parse_hook_pybind11_replace_catch2,
'Qt5': parse_hook_qt5_check_qtwebengine_disable,
Expand Down