-
Notifications
You must be signed in to change notification settings - Fork 591
WW3 in UFS
WW3 is being used in multiple UFS models including the ufs-weather-model, coastal and HAFS. The following information is intended to help developers and users of UFS.
(This is a work in progress, always feel free to submit issues)
Use the routine in ${INPUTDATA_ROOT_WW3}/createmoddefs/creategridfiles.sh where INPUTDATA_ROOT_WW3 is defined by ufs-weather-model/tests/rt.sh
The creategridfiles.sh needs to be copied to a location you can edit. Follow the instructions at the top of the script replacing the needed variables. Run this script, create your new mod_def files and then update your INPUTDATA_ROOT_WW3 to get new mod_def files. More details can be found below.
For the mesh cap, which is currently used by the ufs-weather-model, please see the dev/ufs-weather-model branch of WW3. Most information can be found on https://github.com/ufs-community/ufs-weather-model
Two switches are used depending on if PDLIB=ON (default OFF).
Without PDLIB: model/bin/switch_meshcap
With PDLIB: model/bin/switch_meshcap_pdlib
model/esmf/switch
model/ftn/wmesmfmd.ftn
Activate the appropriate cpp flag in wmesmfmd.ftn, which are all by default disabled:
#define TEST_WMESMFMD___disabled
#define TEST_WMESMFMD_GETIMPORT___disabled
#define TEST_WMESMFMD_SETEXPORT___disabled
#define TEST_WMESMFMD_CREATEIMPGRID___disabled
#define TEST_WMESMFMD_CREATEEXPGRID___disabled
#define TEST_WMESMFMD_SETUPIMPBMSK___disabled
#define TEST_WMESMFMD_CHARNK___disabled
#define TEST_WMESMFMD_ROUGHL___disabled
#define TEST_WMESMFMD_BOTCUR___disabled
#define TEST_WMESMFMD_RADSTR2D___disabled
#define TEST_WMESMFMD_STOKES3D___disabled
#define TEST_WMESMFMD_PSTOKES___disabled
For example, to get field dumps for all export fields set:
#define TEST_WMESMFMD_SETEXPORT
If print_esmf is set to true, for the wave PETS, at the end of the log files PET???.ESMF_LogFile, you will get the following timing information:
20210323 165120.988 INFO PET519 WAV: wtime: timer count time
20210323 165120.988 INFO PET519 WAV: wtime: InitializeP0 1 0.137806E-03
20210323 165120.988 INFO PET519 WAV: wtime: InitializeP1 1 0.118056E+01
20210323 165120.988 INFO PET519 WAV: wtime: InitializeP3 1 0.187102E+00
20210323 165120.988 INFO PET519 WAV: wtime: DataInitialize 2 0.713091E-01
20210323 165120.988 INFO PET519 WAV: wtime: ModelAdvance 96 0.700550E+03
20210323 165120.988 INFO PET519 WAV: wtime: Finalize 1 0.577927E-03
20210323 165120.988 INFO PET519 WAV: wtime: GetImport 97 0.506196E+01
20210323 165120.988 INFO PET519 WAV: wtime: SetExport 97 0.174132E+02
20210323 165120.988 INFO PET519 WAV: wtime: FieldGather 485 0.484256E+01
20210323 165120.988 INFO PET519 WAV: wtime: FieldFill 691 0.661621E-01
(For information about using ESMF profiling tools in ufs-weather-model please see: https://github.com/ufs-community/ufs-weather-model/wiki/Advanced-Topics-for-developers#profiling-timing-across-components)
The manual for the last public release can be found here: https://github.com/NOAA-EMC/WW3/wiki/Tech-Docs or you can build the latest version of the manual from the manual directory. See the section in the appendix titled "Coupling with NUOPC".
New inputs for WW3 for the UFS regression tests are needed when the model updates and there has been an update to mod_def files, or if you would like to update other things, such as time steps, physics settings, add a new grid, etc.
So the first thing to do is to work backwards to find the existing UFS WW3 Inputs:
This variable is called INPUTDATA_ROOT_WW3 in rt.sh see https://github.com/ufs-community/ufs-weather-model/blob/develop/tests/rt.sh#L1037 Note, these inputs will update, so be sure to make sure you have the latest copy from develop.
For whatever platform you are on, you will have to figure out what this full path is.
Then you will copy this entire folder to your directory somewhere.
For example:
cp -r /scratch2/NAGAPE/epic/UFS-WM_RT/NEMSfv3gfs/input-data-20250507/WW3_input_data_20250225 WW3_input_Update_IAUtest
Again, remember this can change over time, so make sure you grabbed the latest version.
INPUTDATA_ROOT_WW3 contains
- a folder called createmoddefs
- mod_def.${grid} files used in RT tests
- rmp_src_*.nc files used for multi-grid tests (should check if these are even still used..)
- wind.natl_6m an input file used for HAFS which is the wind outside of the main domain.
createmoddefs contains
- creategridfiles.sh a shell script that is used to create new mod_defs
- ww3_grid.inp.${grid} input files
- ${grid}.msh mesh files for unstructured grids.
The mod_defs are created by the ww3_grid.inp files in createmoddefs. Update the ww3_grid.inp as required. A few things to remember:
- To couple to MOM6, the inp file should have
&OUTS USSP = 1, IUSSP = 3, STK_WN = 0.04, 0.110, 0.3305 / - The global WW3 time step needs should be an even divider of (or the same as) the coupling time step.
To create new mod_defs, you will use the creategridfiles.sh file. You will need to update a few variables at the top of the script:
#export UFSMODELDIR=<pathto>/ufs-weather-model
#export OUTDIR=<pathtoinputdata>/WW3_input_data
#export SRCDIR=<pathtoinputdata>/WW3_input_data/createmoddefs
#export WORKDIR=<pathtoworkdir>/workdir
Note in these the OUTDIR is the path to where you copied the INPUTDATA_ROOT_WW3 in your local space. WORKDIR is your working directory where intermediate files are saved and log files from the various ww3_grid exes are stored. The UFSMODELDIR is a path to your clone of UFS that you would like to use. Note if you need to update WW3, it should be updated in this path.
Once these are updated run the creategridfiles.sh executable.
Remove your "workdir" and clean up any updates to createmoddefs/creategridfiles.sh It should not point to your local copies, although we sometimes forget to do this.
Don't forget to point to your new files in your rt.sh for the INPUTDATA_ROOT_WW3 while testing your new updates. Address issues as they arrise and when creating a PR to UFS, make sure to mark in the PR template that you have new input data.
Quick Links