Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Expected Differences
If yes, please describe:
Yes. This PR brings in a new make file (
SRC/FEMAIN/Makefile.hip) that can be used to build FastEddy for AMD GPU accelerated platforms. I've also included a small bit of documentation for building FastEddy on AMD GPU systems, using this new make file. These changes do not require any modification to source code inmain_v2.0, as the instructions provided inMakefile.hipleverage ROCm'shipify-clangto convert CUDA code to HIP code prior to compilation withamdclang++/hipcc.Pull Request Testing
I've run the convective boundary layer example on both Nvidia V100 and AMD MI210 systems. @jsauer-NCAR has reviewed model output from both runs and confirmed solutions are in agreement.
Recommend testing for the reviewer(s) to perform, including the location of input datasets, and any additional instructions:
Reviewers are invited to obtain an account on our system to test FastEddy on AMD MI210 GPUs, should they want to run existing tutorials on the AMD platform. Please reach out to me directly to get an account.
Do these changes include sufficient documentation updates, ensuring that no errors or warnings exist in the build of the documentation? [Yes or No]
Yes. However, I could imagine it being worthwhile to include docs on how to run on systems with AMD GPUs (in addition to the build instructions). To this end, we can provide run instructions for our cluster (Galapagos). I might be able to request cycles on Setonix to put together run instructions on Pawsey Supercomputing Centre's Setonix. Down the road, if users or maintainers wish to run on Frontier, we'd be happy to join in on an allocation request for Frontier and help develop build and run documentation specifically for Frontier.
Do these changes include sufficient testing updates? [Yes or No]
No. This is only due to the lack of availability of CI platforms that offer AMD GPUs for testing. Fluid Numerics' Galapagos cluster could do this, but we would need to discuss a support contract to provide GPU-hours dedicated for CI connected via Github Actions and our superci tool.
Will this PR result in changes to the test suite? [Yes or No]
If yes, describe the new output and/or changes to the existing output:
Not presently.
Pull Request Checklist