Skip to content

Build ufs_weather_model on Gaea-C5 + update cubed sphere gitmodules for perturbation/increments for cold starts #2271 #2269

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
zach1221 merged 26 commits into from
Jun 7, 2024
Merged

Build ufs_weather_model on Gaea-C5 + update cubed sphere gitmodules for perturbation/increments for cold starts #2271 #2269

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
8b9f162
update ufs_gaea.intel.lua for building ufs_weather_model on Gaea
DavidBurrows-NCO May 6, 2024
026e2f0
remove commented lines in modulefiles/ufs_gaea.intel.lua
DavidBurrows-NCO May 6, 2024
5f6fdee
update help/whatis in modulefiles/ufs_gaea.intel.lua
DavidBurrows-NCO May 6, 2024
84c31b4
add stack-python load back to ufs_gaea.intel.lua
DavidBurrows-NCO May 7, 2024
db99a8b
Merge https://github.com/ufs-community/ufs-weather-model into gaea_build
DavidBurrows-NCO May 7, 2024
ca16100
unload darshan and cray-libsci in module files for Gaea
DavidBurrows-NCO May 10, 2024
547a435
bring in changes from PR#2258
DavidBurrows-NCO Jun 3, 2024
c51a972
revert spack-stack to 1.5.1 for AWS
DavidBurrows-NCO Jun 4, 2024
fcb2f16
CMakeLists.txt update
DavidBurrows-NCO Jun 4, 2024
b52f050
since ufs_common.lua is used, remove modulefiles/ufs_common4noaacloud…
DavidBurrows-NCO Jun 4, 2024
aaaf393
spack-stack 1.6.0 and revert detect_machine
DavidBurrows-NCO Jun 6, 2024
f43fcbb
point to remote fv3atm
DavidBurrows-NCO Jun 6, 2024
c4eaf09
Merge branch 'develop' into gaea_build
DavidBurrows-NCO Jun 6, 2024
6b5cf6b
update FV3 feature branch hash
jkbk2004 Jun 6, 2024
6b264a6
Update HYCOM hash latest
jkbk2004 Jun 6, 2024
7712e37
[AutoRT] Derecho Job Completed.
epic-cicd-jenkins Jun 6, 2024
db41df5
[AutoRT] Orion Job Completed.
epic-cicd-jenkins Jun 6, 2024
2e7b519
add hercules RT logs: passed
zach1221 Jun 6, 2024
bf66116
add gaea RT log passed
FernandoAndrade-NOAA Jun 6, 2024
41f80ec
add jet RT log passed
FernandoAndrade-NOAA Jun 7, 2024
738a768
[AutoRT] Hera Job Completed.
epic-cicd-jenkins Jun 7, 2024
a53210b
WCOSS2 RT Log: Pass
BrianCurtis-NOAA Jun 7, 2024
12270a7
add control_p8_gnu ORT logs: passed
zach1221 Jun 7, 2024
001455b
add regional_control_gnu ORT logs: passed
zach1221 Jun 7, 2024
b37e931
add cpld_control_gnu ORT logs: passed
zach1221 Jun 7, 2024
bd1d8ad
Update FV3 hash and revert change in gitmodules
jkbk2004 Jun 7, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion FV3
Open in desktop
Submodule FV3 updated from b36400 to 07c26d
2 changes: 1 addition & 1 deletion HYCOM-interface/HYCOM
Open in desktop
Submodule HYCOM updated 1 files
+1 −1 ALT_CODE/README.src.dimensions
1 change: 1 addition & 0 deletions cmake/configure_noaacloud.intel.cmake
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
set(PARALLEL_NETCDF ON CACHE BOOL "Enable parallel NetCDF" FORCE)
13 changes: 6 additions & 7 deletions modulefiles/ufs_gaea.intel.lua
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,7 @@ help([[

whatis([===[Loads libraries needed for building the UFS Weather Model on Gaea ]===])

load("PrgEnv-intel/8.3.3")
load("intel-classic/2023.1.0")
load("cray-mpich/8.1.25")
load("python/3.9.12")
Comment thread
DavidBurrows-NCO marked this conversation as resolved.

prepend_path("MODULEPATH", "/ncrc/proj/epic/spack-stack/spack-stack-1.6.0/envs/unified-env/install/modulefiles/Core")
prepend_path("MODULEPATH", "/ncrc/proj/epic/spack-stack/modulefiles")

stack_intel_ver=os.getenv("stack_intel_ver") or "2023.1.0"
load(pathJoin("stack-intel", stack_intel_ver))
Expand All @@ -22,8 +16,13 @@ load(pathJoin("stack-cray-mpich", stack_cray_mpich_ver))
stack_python_ver=os.getenv("stack_python_ver") or "3.10.13"
load(pathJoin("stack-python", stack_python_ver))

cmake_ver=os.getenv("cmake_ver") or "3.23.1"
load(pathJoin("cmake", cmake_ver))

load("ufs_common")
load("nccmp/1.9.0.1")

nccmp_ver=os.getenv("nccmp_ver") or "1.9.0.1"
load(pathJoin("nccmp", nccmp_ver))

unload("darshan-runtime")
unload("cray-libsci")
Expand Down
48 changes: 24 additions & 24 deletions tests/logs/OpnReqTests_control_p8_hera.log
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
Tue Jun 4 22:17:00 UTC 2024
Thu Jun 6 17:30:07 UTC 2024
Start Operation Requirement Test


baseline dir = /scratch1/NCEPDEV/stmp4/Zachary.Shrader/FV3_OPNREQ_TEST/OPNREQ_TEST/control_p8_bit_base_gnu
working dir = /scratch1/NCEPDEV/stmp2/Zachary.Shrader/FV3_OPNREQ_TEST/opnReqTest_1695475/bit_base_bit_base
working dir = /scratch1/NCEPDEV/stmp2/Zachary.Shrader/FV3_OPNREQ_TEST/opnReqTest_475747/bit_base_bit_base
Checking test bit_base results ....
Moving baseline bit_base files ....
Moving sfcf000.nc .........OK
Expand Down Expand Up @@ -51,14 +51,14 @@ Moving baseline bit_base files ....
Moving RESTART/20210323.060000.sfc_data.tile5.nc .........OK
Moving RESTART/20210323.060000.sfc_data.tile6.nc .........OK

0: The total amount of wall time = 283.395800
0: The maximum resident set size (KB) = 1444564
0: The total amount of wall time = 282.596271
0: The maximum resident set size (KB) = 1434588

Test bit_base PASS


baseline dir = /scratch1/NCEPDEV/stmp4/Zachary.Shrader/FV3_OPNREQ_TEST/OPNREQ_TEST/control_p8_dbg_base_gnu
working dir = /scratch1/NCEPDEV/stmp2/Zachary.Shrader/FV3_OPNREQ_TEST/opnReqTest_1695475/dbg_base_dbg_base
working dir = /scratch1/NCEPDEV/stmp2/Zachary.Shrader/FV3_OPNREQ_TEST/opnReqTest_475747/dbg_base_dbg_base
Checking test dbg_base results ....
Moving baseline dbg_base files ....
Moving sfcf000.nc .........OK
Expand Down Expand Up @@ -106,14 +106,14 @@ Moving baseline dbg_base files ....
Moving RESTART/20210323.060000.sfc_data.tile5.nc .........OK
Moving RESTART/20210323.060000.sfc_data.tile6.nc .........OK

0: The total amount of wall time = 976.634482
0: The maximum resident set size (KB) = 1416500
0: The total amount of wall time = 999.400205
0: The maximum resident set size (KB) = 1425144

Test dbg_base PASS


baseline dir = /scratch1/NCEPDEV/stmp4/Zachary.Shrader/FV3_OPNREQ_TEST/OPNREQ_TEST/control_p8_std_base_gnu
working dir = /scratch1/NCEPDEV/stmp2/Zachary.Shrader/FV3_OPNREQ_TEST/opnReqTest_1695475/dcp_dcp
working dir = /scratch1/NCEPDEV/stmp2/Zachary.Shrader/FV3_OPNREQ_TEST/opnReqTest_475747/dcp_dcp
Checking test dcp results ....
Comparing sfcf000.nc .....USING NCCMP......OK
Comparing sfcf021.nc .....USING NCCMP......OK
Expand Down Expand Up @@ -160,14 +160,14 @@ Checking test dcp results ....
Comparing RESTART/20210323.060000.sfc_data.tile5.nc .....USING NCCMP......OK
Comparing RESTART/20210323.060000.sfc_data.tile6.nc .....USING NCCMP......OK

0: The total amount of wall time = 253.914740
0: The maximum resident set size (KB) = 1420920
0: The total amount of wall time = 255.583364
0: The maximum resident set size (KB) = 1426160

Test dcp PASS


baseline dir = /scratch1/NCEPDEV/stmp4/Zachary.Shrader/FV3_OPNREQ_TEST/OPNREQ_TEST/control_p8_std_base_gnu
working dir = /scratch1/NCEPDEV/stmp2/Zachary.Shrader/FV3_OPNREQ_TEST/opnReqTest_1695475/mpi_mpi
working dir = /scratch1/NCEPDEV/stmp2/Zachary.Shrader/FV3_OPNREQ_TEST/opnReqTest_475747/mpi_mpi
Checking test mpi results ....
Comparing sfcf000.nc .....USING NCCMP......OK
Comparing sfcf021.nc .....USING NCCMP......OK
Expand Down Expand Up @@ -214,14 +214,14 @@ Checking test mpi results ....
Comparing RESTART/20210323.060000.sfc_data.tile5.nc .....USING NCCMP......OK
Comparing RESTART/20210323.060000.sfc_data.tile6.nc .....USING NCCMP......OK

0: The total amount of wall time = 249.447063
0: The maximum resident set size (KB) = 1432868
0: The total amount of wall time = 458.123587
0: The maximum resident set size (KB) = 1425416

Test mpi PASS


baseline dir = /scratch1/NCEPDEV/stmp4/Zachary.Shrader/FV3_OPNREQ_TEST/OPNREQ_TEST/control_p8_std_base_gnu
working dir = /scratch1/NCEPDEV/stmp2/Zachary.Shrader/FV3_OPNREQ_TEST/opnReqTest_1695475/rst_rst
working dir = /scratch1/NCEPDEV/stmp2/Zachary.Shrader/FV3_OPNREQ_TEST/opnReqTest_475747/rst_rst
Checking test rst results ....
Comparing sfcf000.nc .....USING NCCMP......OK
Comparing sfcf021.nc .....USING NCCMP......OK
Expand Down Expand Up @@ -268,14 +268,14 @@ Checking test rst results ....
Comparing RESTART/20210323.060000.sfc_data.tile5.nc .....USING NCCMP......OK
Comparing RESTART/20210323.060000.sfc_data.tile6.nc .....USING NCCMP......OK

0: The total amount of wall time = 248.336854
0: The maximum resident set size (KB) = 1423912
0: The total amount of wall time = 259.258210
0: The maximum resident set size (KB) = 1425588

Test rst PASS


baseline dir = /scratch1/NCEPDEV/stmp4/Zachary.Shrader/FV3_OPNREQ_TEST/OPNREQ_TEST/control_p8_std_base_gnu
working dir = /scratch1/NCEPDEV/stmp2/Zachary.Shrader/FV3_OPNREQ_TEST/opnReqTest_1695475/std_base_std_base
working dir = /scratch1/NCEPDEV/stmp2/Zachary.Shrader/FV3_OPNREQ_TEST/opnReqTest_475747/std_base_std_base
Checking test std_base results ....
Moving baseline std_base files ....
Moving sfcf000.nc .........OK
Expand Down Expand Up @@ -323,14 +323,14 @@ Moving baseline std_base files ....
Moving RESTART/20210323.060000.sfc_data.tile5.nc .........OK
Moving RESTART/20210323.060000.sfc_data.tile6.nc .........OK

0: The total amount of wall time = 252.084050
0: The maximum resident set size (KB) = 1429872
0: The total amount of wall time = 255.559793
0: The maximum resident set size (KB) = 1426800

Test std_base PASS


baseline dir = /scratch1/NCEPDEV/stmp4/Zachary.Shrader/FV3_OPNREQ_TEST/OPNREQ_TEST/control_p8_std_base_gnu
working dir = /scratch1/NCEPDEV/stmp2/Zachary.Shrader/FV3_OPNREQ_TEST/opnReqTest_1695475/thr_thr
working dir = /scratch1/NCEPDEV/stmp2/Zachary.Shrader/FV3_OPNREQ_TEST/opnReqTest_475747/thr_thr
Checking test thr results ....
Comparing sfcf000.nc .....USING NCCMP......OK
Comparing sfcf021.nc .....USING NCCMP......OK
Expand Down Expand Up @@ -377,11 +377,11 @@ Checking test thr results ....
Comparing RESTART/20210323.060000.sfc_data.tile5.nc .....USING NCCMP......OK
Comparing RESTART/20210323.060000.sfc_data.tile6.nc .....USING NCCMP......OK

0: The total amount of wall time = 252.909572
0: The maximum resident set size (KB) = 1408380
0: The total amount of wall time = 254.930047
0: The maximum resident set size (KB) = 1425036

Test thr PASS

OPERATION REQUIREMENT TEST WAS SUCCESSFUL
Tue Jun 4 23:22:43 UTC 2024
Elapsed time: 01h:05m:43s. Have a nice day!
Fri Jun 7 04:02:11 UTC 2024
Elapsed time: 10h:32m:05s. Have a nice day!
24 changes: 12 additions & 12 deletions tests/logs/OpnReqTests_cpld_control_nowave_noaero_p8_hera.log
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
Wed Jun 5 13:31:08 UTC 2024
Fri Jun 7 15:01:46 UTC 2024
Start Operation Requirement Test


baseline dir = /scratch1/NCEPDEV/stmp4/Zachary.Shrader/FV3_OPNREQ_TEST/OPNREQ_TEST/cpld_control_c96_noaero_p8_dbg_base_gnu
working dir = /scratch1/NCEPDEV/stmp2/Zachary.Shrader/FV3_OPNREQ_TEST/opnReqTest_3079325/dbg_base_dbg_base
working dir = /scratch1/NCEPDEV/stmp2/Zachary.Shrader/FV3_OPNREQ_TEST/opnReqTest_1622796/dbg_base_dbg_base
Checking test dbg_base results ....
Moving baseline dbg_base files ....
Moving sfcf021.tile1.nc .........OK
Expand Down Expand Up @@ -66,14 +66,14 @@ Moving baseline dbg_base files ....
Moving RESTART/iced.2021-03-23-21600.nc .........OK
Moving RESTART/ufs.cpld.cpl.r.2021-03-23-21600.nc .........OK

0: The total amount of wall time = 1510.383503
0: The maximum resident set size (KB) = 1529668
0: The total amount of wall time = 1341.667887
0: The maximum resident set size (KB) = 1524488

Test dbg_base PASS


baseline dir = /scratch1/NCEPDEV/stmp4/Zachary.Shrader/FV3_OPNREQ_TEST/OPNREQ_TEST/cpld_control_c96_noaero_p8_std_base_gnu
working dir = /scratch1/NCEPDEV/stmp2/Zachary.Shrader/FV3_OPNREQ_TEST/opnReqTest_3079325/rst_rst
working dir = /scratch1/NCEPDEV/stmp2/Zachary.Shrader/FV3_OPNREQ_TEST/opnReqTest_1622796/rst_rst
Checking test rst results ....
Comparing sfcf021.tile1.nc .....USING NCCMP......OK
Comparing sfcf021.tile2.nc .....USING NCCMP......OK
Expand Down Expand Up @@ -135,14 +135,14 @@ Checking test rst results ....
Comparing RESTART/iced.2021-03-23-21600.nc .....USING NCCMP......OK
Comparing RESTART/ufs.cpld.cpl.r.2021-03-23-21600.nc .....USING NCCMP......OK

0: The total amount of wall time = 517.813920
0: The maximum resident set size (KB) = 1524680
0: The total amount of wall time = 421.484165
0: The maximum resident set size (KB) = 1532072

Test rst PASS


baseline dir = /scratch1/NCEPDEV/stmp4/Zachary.Shrader/FV3_OPNREQ_TEST/OPNREQ_TEST/cpld_control_c96_noaero_p8_std_base_gnu
working dir = /scratch1/NCEPDEV/stmp2/Zachary.Shrader/FV3_OPNREQ_TEST/opnReqTest_3079325/std_base_std_base
working dir = /scratch1/NCEPDEV/stmp2/Zachary.Shrader/FV3_OPNREQ_TEST/opnReqTest_1622796/std_base_std_base
Checking test std_base results ....
Moving baseline std_base files ....
Moving sfcf021.tile1.nc .........OK
Expand Down Expand Up @@ -205,11 +205,11 @@ Moving baseline std_base files ....
Moving RESTART/iced.2021-03-23-21600.nc .........OK
Moving RESTART/ufs.cpld.cpl.r.2021-03-23-21600.nc .........OK

0: The total amount of wall time = 430.945213
0: The maximum resident set size (KB) = 1513244
0: The total amount of wall time = 396.603624
0: The maximum resident set size (KB) = 1524764

Test std_base PASS

OPERATION REQUIREMENT TEST WAS SUCCESSFUL
Wed Jun 5 14:37:12 UTC 2024
Elapsed time: 01h:06m:04s. Have a nice day!
Fri Jun 7 16:08:56 UTC 2024
Elapsed time: 01h:07m:10s. Have a nice day!
30 changes: 15 additions & 15 deletions tests/logs/OpnReqTests_regional_control_hera.log
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
Wed Jun 5 02:54:52 UTC 2024
Fri Jun 7 12:29:07 UTC 2024
Start Operation Requirement Test


baseline dir = /scratch1/NCEPDEV/stmp4/Zachary.Shrader/FV3_OPNREQ_TEST/OPNREQ_TEST/regional_control_bit_base_gnu
working dir = /scratch1/NCEPDEV/stmp2/Zachary.Shrader/FV3_OPNREQ_TEST/opnReqTest_836548/bit_base_bit_base
working dir = /scratch1/NCEPDEV/stmp2/Zachary.Shrader/FV3_OPNREQ_TEST/opnReqTest_599192/bit_base_bit_base
Checking test bit_base results ....
Moving baseline bit_base files ....
Moving dynf000.nc .........OK
Expand All @@ -15,14 +15,14 @@ Moving baseline bit_base files ....
Moving NATLEV.GrbF00 .........OK
Moving NATLEV.GrbF06 .........OK

0: The total amount of wall time = 1240.233579
0: The maximum resident set size (KB) = 738044
0: The total amount of wall time = 1373.672576
0: The maximum resident set size (KB) = 750196

Test bit_base PASS


baseline dir = /scratch1/NCEPDEV/stmp4/Zachary.Shrader/FV3_OPNREQ_TEST/OPNREQ_TEST/regional_control_std_base_gnu
working dir = /scratch1/NCEPDEV/stmp2/Zachary.Shrader/FV3_OPNREQ_TEST/opnReqTest_836548/dcp_dcp
working dir = /scratch1/NCEPDEV/stmp2/Zachary.Shrader/FV3_OPNREQ_TEST/opnReqTest_599192/dcp_dcp
Checking test dcp results ....
Comparing dynf000.nc .....USING NCCMP......OK
Comparing dynf006.nc .....USING NCCMP......OK
Expand All @@ -33,14 +33,14 @@ Checking test dcp results ....
Comparing NATLEV.GrbF00 .....USING CMP......OK
Comparing NATLEV.GrbF06 .....USING CMP......OK

0: The total amount of wall time = 2160.002829
0: The maximum resident set size (KB) = 712880
0: The total amount of wall time = 2172.451269
0: The maximum resident set size (KB) = 725616

Test dcp PASS


baseline dir = /scratch1/NCEPDEV/stmp4/Zachary.Shrader/FV3_OPNREQ_TEST/OPNREQ_TEST/regional_control_std_base_gnu
working dir = /scratch1/NCEPDEV/stmp2/Zachary.Shrader/FV3_OPNREQ_TEST/opnReqTest_836548/std_base_std_base
working dir = /scratch1/NCEPDEV/stmp2/Zachary.Shrader/FV3_OPNREQ_TEST/opnReqTest_599192/std_base_std_base
Checking test std_base results ....
Moving baseline std_base files ....
Moving dynf000.nc .........OK
Expand All @@ -52,14 +52,14 @@ Moving baseline std_base files ....
Moving NATLEV.GrbF00 .........OK
Moving NATLEV.GrbF06 .........OK

0: The total amount of wall time = 2262.226051
0: The maximum resident set size (KB) = 711912
0: The total amount of wall time = 2258.515223
0: The maximum resident set size (KB) = 723052

Test std_base PASS


baseline dir = /scratch1/NCEPDEV/stmp4/Zachary.Shrader/FV3_OPNREQ_TEST/OPNREQ_TEST/regional_control_std_base_gnu
working dir = /scratch1/NCEPDEV/stmp2/Zachary.Shrader/FV3_OPNREQ_TEST/opnReqTest_836548/thr_thr
working dir = /scratch1/NCEPDEV/stmp2/Zachary.Shrader/FV3_OPNREQ_TEST/opnReqTest_599192/thr_thr
Checking test thr results ....
Comparing dynf000.nc .....USING NCCMP......OK
Comparing dynf006.nc .....USING NCCMP......OK
Expand All @@ -70,11 +70,11 @@ Checking test thr results ....
Comparing NATLEV.GrbF00 .....USING CMP......OK
Comparing NATLEV.GrbF06 .....USING CMP......OK

0: The total amount of wall time = 2237.695388
0: The maximum resident set size (KB) = 699580
0: The total amount of wall time = 2224.886814
0: The maximum resident set size (KB) = 725384

Test thr PASS

OPERATION REQUIREMENT TEST WAS SUCCESSFUL
Wed Jun 5 05:20:18 UTC 2024
Elapsed time: 02h:25m:27s. Have a nice day!
Fri Jun 7 15:01:07 UTC 2024
Elapsed time: 02h:32m:00s. Have a nice day!
Loading