Changes to move SRW app towards NCO complaince

[danielabdi-noaa]

DESCRIPTION OF CHANGES:

Some changes needed for NCO standard compliance.

• option to build directly to an exec directory instead of bin
• option to build each of the components of SRW app (currently five) separately or in combination.

TARGETS
   default = builds the default list of apps (also not passing any target does the same)
   all = builds all apps
   Or any combinations of (ufs, ufs_utils, upp, gsi, rrfs_utils)

I don't think a separate script for each of them is needed especially because you can build two of them at a time
with the latter approach which the former can't do.

./devbuild.sh -p=hera           # build default list
./devbuild.sh -p=hera default   # same as default
./devbuild.sh -p=hera all       # build everything
./devbuild.sh -p=hera ufs       # build only ufs weather model
./devbuild.sh -p=hera ufs_utils # build only ufs_utils
./devbuild.sh -p=hera upp       # build only upp
./devbuild.sh -p=hera gsi       # build only gsi
./devbuild.sh -p=hera gsi rrfs_utils   # build only gsi and rrfs_utils 
                                       # rrfs_utils needs gsi to be built first

• renamed src to sorc along with necessary changes in Externals.cfg
• Added options --clean, --build and --install to do separate build and install

TESTS CONDUCTED:

Run one test locally on hera, will run automated test

DEPENDENCIES:

Currently none, but will need to modify regional_workflow when default bin directly is changed back to exec

DOCUMENTATION:

Currently none, but changes in devbuild.sh and directory structure need to be documented later.

[venitahagerty]

Machine: hera
Compiler: intel
Job: WE
Repo location: /scratch1/BMC/zrtrr/rrfs_ci/autoci/pr/1025708072/20220814105016/ufs-srweather-app
Build was Successful
Rocoto jobs started
Long term tracking will be done on 9 experiments
If test failed, please make changes and add the following label back:
ci-hera-intel-WE
Experiment Succeeded on hera: grid_RRFS_CONUS_25km_ics_GSMGFS_lbcs_GSMGFS_suite_GFS_v15p2
Experiment Succeeded on hera: grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_RAP_suite_HRRR
Experiment Succeeded on hera: grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_RAP_suite_RRFS_v1beta
Experiment Succeeded on hera: grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_RAP_suite_HRRR
Experiment Succeeded on hera: grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v15p2
Experiment Succeeded on hera: nco_grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_RAP_suite_HRRR
Experiment Succeeded on hera: grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16
Experiment Succeeded on hera: grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_HRRR_suite_RRFS_v1beta
Experiment Succeeded on hera: grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_HRRR_suite_HRRR
All experiments completed

[christinaholtNOAA]

@danielabdi-noaa I think this is a nice, clean implementation of the NCO standards that we were looking for. I left just a few comments/questions below.

[christinaholtNOAA]

Is this the magic that separates the build and install steps?

Are the names of the targets arbitrary? I suppose I'm curious why mbuild and minstall instead of just build and install.

[danielabdi-noaa]

Yes, but the cause of the problem was the external projects add. You would expect a make all would just build the binaries and not install but default behavior of ExternalProject_Add is non-intiutive -- it builds and installs on a make all. So I added STEP_TARGETS build install to separate them and then connect to a custom target at the top level. I think future CMake version could potentially simplify this. I used mbuild and minstall because I couldn't use "install" as target since it is a reserved keyword. "build" was fine though but changed it to be consistent.

[danielabdi-noaa]

I have removed the custom install target because there was already one anyway, so we have a build target instead of mbuild now. Still couldn't find a solution to prevent make all from installing the project...

[christinaholtNOAA]

Are these the only two options? Could this be set arbitrarily?

[danielabdi-noaa]

Unfortunately no. UPP and UFS_UTILS do not use CMAKE_INSTALL_BINDIR as the destination target, instead they set to either "bin" or "exec" where the latter is used if an "EMC_EXEC_DIR" variable is turned on. But once these are fixed, we should be able to install to any bin directory name.

[christinaholtNOAA]

Does this --clean behave the same as it did before? Should we have both --clean and --remove?

[danielabdi-noaa]

Thanks for catching that bug! --clean does a make clean on the target you selected. I am not sure if --remove that does rm -rf build is needed now but I kept it just in case.

[chan-hoo]

@danielabdi-noaa, "./devbuild.sh -p=hera --bin-dir=exec upp" doesn't work correctly. The executable "upp.x" is not created in the "exec" directory but in "bin" directory.

[danielabdi-noaa]

@chan-hoo Hmm that is weird. It does put the upp.x executable in the exec directory for me. Maybe removing the build directory first can help?

$ ./devbuild.sh -p=hera --bin-dir=exec upp
....
[ 89%] Built target upp
[100%] Built target upp.x
Install the project...
-- Install configuration: "Release"
-- Installing: /scratch2/BMC/gsd-hpcs/Daniel.Abdi/ufs-srweather-app/exec/upp.x
-- Up-to-date: /scratch2/BMC/gsd-hpcs/Daniel.Abdi/ufs-srweather-app/include
$ ls exec
ufs_srweather_app.settings  upp.x

[chan-hoo]

@danielabdi-noaa, I don't know what happened. I removed the 'build' directory before I tested. What I got:
"CMake Warning:
Manually-specified variables were not used by the project:
EMC_EXEC_DIR"

[Chan-hoo.Jeon@hfe09 exec]$ ls
ufs_srweather_app.settings

[Chan-hoo.Jeon@hfe09 ufs-srweather-app]$ cd bin
[Chan-hoo.Jeon@hfe09 bin]$ ls
upp.x

[danielabdi-noaa]

@chan-hoo It looks like a recent PR has removed "EMC_EXEC_DIR" option which is what is causing the problem.
NOAA-EMC/UPP@8767929#diff-1e7de1ae2d059d21e1dd75d5812d5a34b0222cef273b7c3a2af62eb747f9d20a
I had a couple of days older UPP which was why I was not seeing it.
This is a good thing because now we can now build not only to "bin" and "exec" but any arbitrary exe directory.
UFS_UTILS is the only one that uses this hack but it have been removed too so I will check ...

[danielabdi-noaa]

@chan-hoo Could you please try again? I have updated for the UPP change regarding removal of EMC_EXEC_DIR=on/off.
Now it uses "exec_dir" to specifiy the excutable directory which is unnecssary in my opnion. There is alread a CMAKE_INSTALL_BINDIR for that purpose which GSI and others use. UFS_UTILS still uses "EMC_EXEC_DIR" so that still needs to be removed for us to build to any executable directory name.

[chan-hoo]

@danielabdi-noaa, It works well now. Thank you for the quick fix!

[danielabdi-noaa]

@chan-hoo Thanks for testing! I have created a PR to modify UFS_UTILS installation directory to be anything aside from bin and exec ufs-community/UFS_UTILS#685

[panll]

Tested on Hera. Works fine. Thanks!