Skip to content

Fix use of CUDA CCs in LLVM easyblock and cleanup #3755

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 9 commits into
base: develop
Choose a base branch
from

Conversation

Flamefire
Copy link
Contributor

@Flamefire Flamefire commented Jun 4, 2025

(created using eb --new-pr)

Combined #3812 & #3815 with additional fixes. Total:

  • Rename usepolly to use_polly consistent with other options
  • Don't warn for mod files for non-minimal builds
  • Use get_cuda_cc_template_value and list_to_cmake_arg to avoid comments and simplify code
  • Reduce use of runtimes_cmake_args (Variables prefixed with e.g. LIBOMPTARGET get passed directly and should rather be passed to the main configure)
  • Fix/Enhance some comments
  • refactorings to super() call, f-Strings and list changes

Additional:

  • Add warning for unknown CPU arch (from CLang easyblock)
  • Relax condition in build_openmp_offload for passing CUDA CCs to allow use with LLVM 18 (i.e. LIBOMPTARGET_DEVICE_ARCHITECTURES is now set for LLVM 18 too, since LLVM 17.0 this is defaulted to "all" in CMakeLists.txt)
  • Refactor and fix check for OpenMP files in sanity check (temporary variable was assigned to final variable inside nvptx_target_cond block)

Test easyconfigs: easybuilders/easybuild-easyconfigs#23028

Fixes #3758

Comment on lines 690 to 691
if gpu_archs:
self.runtimes_cmake_args['LIBOMPTARGET_DEVICE_ARCHITECTURES'] = '%s' % '|'.join(gpu_archs)
self._cmakeopts['LIBOMPTARGET_DEVICE_ARCHITECTURES'] = ';'.join(gpu_archs)
Copy link
Contributor

@Thyre Thyre Jun 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is going to work, and I would be extremely hesitant towards reverting this.
The sanity check may pass, simply because all architectures are being built, but this is not what we want.

See e.g. #3675 (comment) for LLVM 19. I think this is also something where @bedroge had issues with, which we resolved with the currently implemented approach.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually see the opposite: I did a test with the CLang and the LLVM easyblock of the "same" easyconfig and did a diff:

Nur in /tmp/install-old/software/Clang/18.1.8-GCCcore-13.3.0-CUDA-12.6.0/lib64: libomptarget-nvptx-sm_35.bc.
Nur in /tmp/install-old/software/Clang/18.1.8-GCCcore-13.3.0-CUDA-12.6.0/lib64: libomptarget-nvptx-sm_37.bc.
Nur in /tmp/install-old/software/Clang/18.1.8-GCCcore-13.3.0-CUDA-12.6.0/lib64: libomptarget-nvptx-sm_50.bc.
Nur in /tmp/install-old/software/Clang/18.1.8-GCCcore-13.3.0-CUDA-12.6.0/lib64: libomptarget-nvptx-sm_52.bc.
Nur in /tmp/install-old/software/Clang/18.1.8-GCCcore-13.3.0-CUDA-12.6.0/lib64: libomptarget-nvptx-sm_53.bc.
Nur in /tmp/install-old/software/Clang/18.1.8-GCCcore-13.3.0-CUDA-12.6.0/lib64: libomptarget-nvptx-sm_60.bc.
Nur in /tmp/install-old/software/Clang/18.1.8-GCCcore-13.3.0-CUDA-12.6.0/lib64: libomptarget-nvptx-sm_61.bc.
Nur in /tmp/install-old/software/Clang/18.1.8-GCCcore-13.3.0-CUDA-12.6.0/lib64: libomptarget-nvptx-sm_62.bc.
Nur in /tmp/install-old/software/Clang/18.1.8-GCCcore-13.3.0-CUDA-12.6.0/lib64: libomptarget-nvptx-sm_70.bc.
Nur in /tmp/install-old/software/Clang/18.1.8-GCCcore-13.3.0-CUDA-12.6.0/lib64: libomptarget-nvptx-sm_72.bc.
Nur in /tmp/install-old/software/Clang/18.1.8-GCCcore-13.3.0-CUDA-12.6.0/lib64: libomptarget-nvptx-sm_75.bc.
Nur in /tmp/install-old/software/Clang/18.1.8-GCCcore-13.3.0-CUDA-12.6.0/lib64: libomptarget-nvptx-sm_86.bc.
Nur in /tmp/install-old/software/Clang/18.1.8-GCCcore-13.3.0-CUDA-12.6.0/lib64: libomptarget-nvptx-sm_87.bc.
Nur in /tmp/install-old/software/Clang/18.1.8-GCCcore-13.3.0-CUDA-12.6.0/lib64: libomptarget-nvptx-sm_89.bc.
Nur in /tmp/install-old/software/Clang/18.1.8-GCCcore-13.3.0-CUDA-12.6.0/lib64: libomptarget-nvptx-sm_90.bc.

I inspected the CMake files and every CMake value with prefix LIBOMPTARGET get automatically passed through to the sub-builds.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There were significant changes with LLVM 19 & LLVM 20, which might influence the results a bit.
I just checked my LLVM 19 & 20 installs on my machines, and with the currently implemented approach, only the passed architectures are built.

I don't have an LLVM 18.1.8 installation with the new EasyBlock available to compare to, and unfortunately jsc-zen3 hasn't either. Maybe we missed something for this version in particular.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made a companion PR for testing with some easyconfigs of various versions. Build is running, will check

IIRC LLVM 20 doesn't use those anymore.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checking a Clang installation done with the old EasyBlock just before EB 5.0.0 (4.9.5dev), I can see that all architectures are built:

jreuter@ZAM054  /opt/EasyBuild/apps/software/Clang/18.1.8-GCCcore-13.3.0-CUDA-12.6.0  find . -name "*.bc"
./lib/libomptarget-amdgpu-gfx1150.bc
./lib/libomptarget-amdgpu-gfx900.bc
./lib/libomptarget-amdgpu-gfx941.bc
./lib/libomptarget-amdgpu-gfx1032.bc
./lib/libomptarget-amdgpu-gfx902.bc
./lib/libomptarget-nvptx-sm_89.bc
./lib/libomptarget-amdgpu-gfx801.bc
./lib/libomptarget-amdgpu-gfx1103.bc
./lib/libomptarget-amdgpu-gfx1010.bc
./lib/libomptarget-nvptx-sm_37.bc
./lib/libomptarget-nvptx-sm_80.bc
./lib/libomptarget-nvptx-sm_86.bc
./lib/libomptarget-nvptx-sm_61.bc
./lib/libomptarget-nvptx-sm_70.bc
./lib/libomptarget-nvptx-sm_50.bc
./lib/libomptarget-amdgpu-gfx1030.bc
./lib/libomptarget-nvptx-sm_62.bc
./lib/libomptarget-nvptx-sm_75.bc
./lib/libomptarget-nvptx-sm_72.bc
./lib/libomptarget-amdgpu-gfx1031.bc
./lib/libomptarget-amdgpu-gfx90c.bc
./lib/libomptarget-nvptx-sm_87.bc
./lib/libomptarget-nvptx-sm_60.bc
./lib/libomptarget-amdgpu-gfx1035.bc
./lib/libomptarget-amdgpu-gfx803.bc
./lib/libomptarget-nvptx-sm_35.bc
./lib/libomptarget-nvptx-sm_53.bc
./lib/libomptarget-amdgpu-gfx942.bc
./lib/libomptarget-amdgpu-gfx940.bc
./lib/libomptarget-amdgpu-gfx1101.bc
./lib/libomptarget-amdgpu-gfx1100.bc
./lib/libomptarget-amdgpu-gfx90a.bc
./lib/libomptarget-amdgpu-gfx906.bc
./lib/libomptarget-amdgpu-gfx701.bc
./lib/libomptarget-amdgpu-gfx700.bc
./lib/libomptarget-amdgpu-gfx908.bc
./lib/libomptarget-amdgpu-gfx1033.bc
./lib/libomptarget-amdgpu-gfx1151.bc
./lib/libomptarget-amdgpu-gfx1034.bc
./lib/libomptarget-amdgpu-gfx1036.bc
./lib/libomptarget-nvptx-sm_90.bc
./lib/libomptarget-amdgpu-gfx1102.bc
./lib/libomptarget-nvptx-sm_52.bc

Compared to LLVM 19.1.1 with the latest EasyBlock:

 jreuter@ZAM054  /opt/EasyBuild/apps/software/LLVM/19.1.1-GCCcore-13.3.0  find . -name "*.bc"
./lib/x86_64-unknown-linux-gnu/libomptarget-nvptx-sm_80.bc
./lib/x86_64-unknown-linux-gnu/libomptarget-nvptx-sm_75.bc

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. I might have made a mistake during the comparison. I'm doing a rebuild of both versions now.

The idea was to simplify this and make it easier to understand in the future. To me this runtimes_cmake_args thing was highly suspicious/confusing especially with the pipe separator and it isn't clear either if and why this argument isn't required for the "main" build.
In the source you can see that there are some variables
and the LIBOMP* prefix passed through by default. And according to git blame this has been virtually always the case.
So I'm confident this is the best way, especially as I couldn't find any examples doing it the current way which makes me thing this is "safer".

I think building a few versions (see the EC PR) with this should be enough to check that this works for all 3 major versions.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I checked LLVM 18.1.8 with your changes, and there it seems to work with your solution as well.

$ find . -name "*.bc"
./lib/libomptarget-nvptx-sm_75.bc
./lib/libomptarget-amdgpu-gfx1101.bc
./lib/libomptarget-nvptx-sm_80.bc

Lets check the other ones for safety as well.

Copy link
Contributor Author

@Flamefire Flamefire Jun 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've redone the tests with Clang 18.1.8 once with the clang easyblock and once with the updated llvm easyblock, both with CCC=7.0,8.0, and can still see that the clang easyblock builds all architectures. Diff of the install folder restricted to added files and ignoring the lib64 symlink:

Only in Clang-Clang/easybuild/reprod/easyblocks: clang.py.
Only in Clang-Clang/lib: libgomp.so.
Only in Clang-Clang/lib: libiomp5.so.
Only in Clang-Clang/lib: libomptarget-amdgpu-gfx1010.bc.
Only in Clang-Clang/lib: libomptarget-amdgpu-gfx1030.bc.
Only in Clang-Clang/lib: libomptarget-amdgpu-gfx1031.bc.
Only in Clang-Clang/lib: libomptarget-amdgpu-gfx1032.bc.
Only in Clang-Clang/lib: libomptarget-amdgpu-gfx1033.bc.
Only in Clang-Clang/lib: libomptarget-amdgpu-gfx1034.bc.
Only in Clang-Clang/lib: libomptarget-amdgpu-gfx1035.bc.
Only in Clang-Clang/lib: libomptarget-amdgpu-gfx1036.bc.
Only in Clang-Clang/lib: libomptarget-amdgpu-gfx1100.bc.
Only in Clang-Clang/lib: libomptarget-amdgpu-gfx1101.bc.
Only in Clang-Clang/lib: libomptarget-amdgpu-gfx1102.bc.
Only in Clang-Clang/lib: libomptarget-amdgpu-gfx1103.bc.
Only in Clang-Clang/lib: libomptarget-amdgpu-gfx1150.bc.
Only in Clang-Clang/lib: libomptarget-amdgpu-gfx1151.bc.
Only in Clang-Clang/lib: libomptarget-amdgpu-gfx700.bc.
Only in Clang-Clang/lib: libomptarget-amdgpu-gfx701.bc.
Only in Clang-Clang/lib: libomptarget-amdgpu-gfx801.bc.
Only in Clang-Clang/lib: libomptarget-amdgpu-gfx803.bc.
Only in Clang-Clang/lib: libomptarget-amdgpu-gfx900.bc.
Only in Clang-Clang/lib: libomptarget-amdgpu-gfx902.bc.
Only in Clang-Clang/lib: libomptarget-amdgpu-gfx906.bc.
Only in Clang-Clang/lib: libomptarget-amdgpu-gfx908.bc.
Only in Clang-Clang/lib: libomptarget-amdgpu-gfx90a.bc.
Only in Clang-Clang/lib: libomptarget-amdgpu-gfx90c.bc.
Only in Clang-Clang/lib: libomptarget-amdgpu-gfx940.bc.
Only in Clang-Clang/lib: libomptarget-amdgpu-gfx941.bc.
Only in Clang-Clang/lib: libomptarget-amdgpu-gfx942.bc.
Only in Clang-Clang/lib: libomptarget-nvptx-sm_35.bc.
Only in Clang-Clang/lib: libomptarget-nvptx-sm_37.bc.
Only in Clang-Clang/lib: libomptarget-nvptx-sm_50.bc.
Only in Clang-Clang/lib: libomptarget-nvptx-sm_52.bc.
Only in Clang-Clang/lib: libomptarget-nvptx-sm_53.bc.
Only in Clang-Clang/lib: libomptarget-nvptx-sm_60.bc.
Only in Clang-Clang/lib: libomptarget-nvptx-sm_61.bc.
Only in Clang-Clang/lib: libomptarget-nvptx-sm_62.bc.
Only in Clang-Clang/lib: libomptarget-nvptx-sm_72.bc.
Only in Clang-Clang/lib: libomptarget-nvptx-sm_75.bc.
Only in Clang-Clang/lib: libomptarget-nvptx-sm_86.bc.
Only in Clang-Clang/lib: libomptarget-nvptx-sm_87.bc.
Only in Clang-Clang/lib: libomptarget-nvptx-sm_89.bc.
Only in Clang-Clang/lib: libomptarget-nvptx-sm_90.bc.
Only in Clang-LLVM/bin: count.
Only in Clang-LLVM/bin: FileCheck.
Only in Clang-LLVM/bin: lli-child-target.
Only in Clang-LLVM/bin: llvm-jitlink-executor.
Only in Clang-LLVM/bin: llvm-PerfectShuffle.
Only in Clang-LLVM/bin: not.
Only in Clang-LLVM/bin: obj2yaml.
Only in Clang-LLVM/bin: split-file.
Only in Clang-LLVM/bin: UnicodeNameMappingGenerator.
Only in Clang-LLVM/bin: yaml2obj.
Only in Clang-LLVM/bin: yaml-bench.
Only in Clang-LLVM/easybuild/reprod/easyblocks: llvm.py.
Only in Clang-LLVM/include: x86_64-pc-linux.
Only in Clang-LLVM/include: x86_64-pc-linux-gnu.
Only in Clang-LLVM/lib/clang/18/lib: x86_64-pc-linux.
Only in Clang-LLVM/lib/clang/18/lib: x86_64-pc-linux-gnu.
Only in Clang-LLVM/lib: libLLVM-18.so.
Only in Clang-LLVM/lib: libLLVM.so.
Only in Clang-LLVM/lib: libLLVM.so.18.1.
Only in Clang-LLVM/lib: libMLIR.so.
Only in Clang-LLVM/lib: libMLIR.so.18.1.
Only in Clang-LLVM/lib: x86_64-pc-linux.
Only in Clang-LLVM/lib: x86_64-pc-linux-gnu.

libgomp and libiomp are symlinks to libomp. Not sure why LLVM doesn't create them

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might worth checking if this has some influence:

Nur in Clang-Clang/lib: libgomp.so.
Nur in Clang-Clang/lib: libiomp5.so.

One can select the OpenMP library with -fopenmp=[...] IIRC.
If choosing libgomp still works, I wouldn't care that the files are missing.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That option seemed to have been considered in an ancient Clang version but dropped. Given that those are symlinks I don't see a reason to not install them just in case.

Comment on lines 501 to 505
self.runtimes_cmake_args['LIBOMPTARGET_PLUGINS_TO_BUILD'] = '%s' % '|'.join(self.offload_targets)
self._cmakeopts['LIBOMPTARGET_PLUGINS_TO_BUILD'] = ';'.join(self.offload_targets)
Copy link
Contributor

@Thyre Thyre Jun 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See other comment.
I don't see why we should revert this when the current approach is proven to work just fine.

We've tested this very extensively already, both with CUDA & ROCm

Copy link
Contributor

@Thyre Thyre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regarding offload targets, see above.
The remainder looks good to me at a quick first glance ( except the failing CI of course 😄 )

@Thyre
Copy link
Contributor

Thyre commented Jun 4, 2025

Test report by @Thyre

Overview of tested easyconfigs (in order)

Build succeeded for 0 out of 1 (1 easyconfigs in total)
zam226 - Linux Ubuntu 22.04, x86_64, 12th Gen Intel(R) Core(TM) i7-12700, Python 3.10.12
See https://gist.github.com/Thyre/28a9e078282f4a68ac8b3061c00ca6f0 for a full test report.


The CMake option changes break builds where offload architectures need to be passed.

@Flamefire
Copy link
Contributor Author

I had messed up quoting one argument. Shall we introduce a function that does "'%s'" % ';'.join(some-list) e.g. as CMakeMake::list_to_cmake_arg?

@Flamefire
Copy link
Contributor Author

As for the CI failure: I'd rather make the function more flexible and use that: easybuilders/easybuild-framework#4913

IMO this is better than having to repeat our logic of determining the CCCs in many places. I had set the build option locally which is why I didn't notice it.
Objections?

@Thyre
Copy link
Contributor

Thyre commented Jun 4, 2025

Both sound like a good idea 👍

@Flamefire Flamefire changed the title Fix use of CUDA CCs in LLVM easyblock and cleanup Clean up LLVM easyblock Jun 4, 2025
@Flamefire Flamefire changed the title Clean up LLVM easyblock Fix use of CUDA CCs in LLVM easyblock and cleanup Jun 4, 2025
@boegel
Copy link
Member

boegel commented Jun 7, 2025

@Flamefire Can you look into resolving the merge conflict after the merge of #3741 ?

@boegel boegel added this to the next release (5.1.1?) milestone Jun 18, 2025
@Flamefire Flamefire force-pushed the 20250604092520_new_pr_llvm branch from 2041060 to 19db109 Compare June 24, 2025 09:35
@Flamefire
Copy link
Contributor Author

Flamefire commented Jun 24, 2025

Rebased on develop after merged PRs

Currently seeing "cannot determine nvptx64 architecture": Child timed out errors in nvidia related tests on a system with H100 GPUs.
I'm quite sure that is not related to this MR and suspect the tests are not expecting a non-CUDA build with GPUs present

@Flamefire
Copy link
Contributor Author

Test report by @Flamefire

Overview of tested easyconfigs (in order)

  • SUCCESS LLVM-19.1.7-GCCcore-13.3.0.eb
  • SUCCESS LLVM-20.1.5-GCCcore-13.3.0.eb

Build succeeded for 2 out of 2 (2 easyconfigs in total)
n1657 - Linux RHEL 8.9 (Ootpa), x86_64, Intel(R) Xeon(R) Platinum 8470 (sapphirerapids), Python 3.9.18
See https://gist.github.com/Flamefire/6e331f57c9d1e776cc44e17422d86efb for a full test report.

@Flamefire
Copy link
Contributor Author

Test report by @Flamefire

Overview of tested easyconfigs (in order)

Build succeeded for 1 out of 2 (2 easyconfigs in total)
c14 - Linux AlmaLinux 9.4, x86_64, AMD EPYC 9334 32-Core Processor (zen4), 4 x NVIDIA NVIDIA H100, 560.35.03, Python 3.9.18
See https://gist.github.com/Flamefire/aac0d936c684e0b3ed4ad8f4bf046153 for a full test report.

@boegel
Copy link
Member

boegel commented Jun 30, 2025

@Flamefire What's up with the failed tests, are those triggered by the changes in this PR? (If not, let's follow up on those in a subsequent PR)

@Flamefire
Copy link
Contributor Author

I though they were not as I couldn't find what was wrong. But it seems they are triggered by this PR as my cross-check test on the same node went through: #3770 (comment)

I'll check again...

@@ -234,16 +235,22 @@ def __init__(self, *args, **kwargs):
"""Initialize LLVM-specific variables."""
super().__init__(*args, **kwargs)

if self.cfg['usepolly'] is not None:
self.log.deprecated("Use of easyconfig parameter 'usepolly', replace by 'use_polly'", '6.0')
self.cfg['use_polly'] = self.cfg['usepolly']
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should only be done if use_polly is not specified. If both are set, use_polly value should be used

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In which case would both be set? That would IMO only be done in the easyconfig in which case I don't see an indication on which should be preferred over the other, if they'd be different in the first place.

use_polly has a default of False, so for this change we'd need to change it to None which makes the documentation worse (but should work without additional changes). So I would just ignore if both are set and keep the current behavior. Or if we handle it, I'd make it an error if both are set, due to the above reason.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we deprecate usepolly, then it should never override the value set in use_polly.
If that means using None as default value for use_polly, so be it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Flamefire Flamefire force-pushed the 20250604092520_new_pr_llvm branch from 9d75652 to b740bfb Compare June 30, 2025 15:25
@Flamefire
Copy link
Contributor Author

I factored out 2 other PRs to make this smaller. and rebased this on them

@Flamefire Flamefire force-pushed the 20250604092520_new_pr_llvm branch 2 times, most recently from 86e11ec to 631bb23 Compare July 1, 2025 14:09
@Flamefire
Copy link
Contributor Author

Flamefire commented Jul 1, 2025

With LLVM-18.1.8-GCCcore-13.3.0.eb I'm running into failures during the test step:

652 on Sapphire Rapids, 326 on AMD EPYC with H100 GPUs (exactly half!)

This is already happening with #3812 which shouldn't cause any such failures. Retesting with develop branch

Build looks fine on AMD EPYC (CPU node)

@boegel
Copy link
Member

boegel commented Jul 1, 2025

Test report by @boegel

Overview of tested easyconfigs (in order)

  • SUCCESS LLVM-18.1.8-GCCcore-13.3.0.eb

Build succeeded for 1 out of 1 (1 easyconfigs in total)
node3512.doduo.os - Linux RHEL 9.4, x86_64, AMD EPYC 7552 48-Core Processor (zen2), Python 3.9.18
See https://gist.github.com/boegel/ba92756f32225c68f246b5cc2d8b4c4d for a full test report.

@boegel
Copy link
Member

boegel commented Jul 2, 2025

@Flamefire this needs a rebase now that #3812 is merged?

@Flamefire Flamefire force-pushed the 20250604092520_new_pr_llvm branch from 631bb23 to b0e4f75 Compare July 2, 2025 10:48
@Flamefire Flamefire force-pushed the 20250604092520_new_pr_llvm branch from b0e4f75 to d56e66a Compare July 2, 2025 12:22
@Flamefire
Copy link
Contributor Author

Rebase only to replace the elif to if as discussed at #3815 (review)

@boegel
Copy link
Member

boegel commented Jul 2, 2025

I'll get #3781 and #3815 merged first, so this can be rebased afterwards...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Bug when building multiple versions of LLVM in one call to easybuild
3 participants