Skip to content

Fixes for reproducibility, failed builds, and CI. More hrrr tests, sh… #17

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
HelinWei-NOAA merged 1 commit into from
Jun 17, 2022
Merged

Fixes for reproducibility, failed builds, and CI. More hrrr tests, sh… #17

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion FV3
Open in desktop
Submodule FV3 updated from e2d12a to 9771e5
835 changes: 590 additions & 245 deletions tests/RegressionTests_cheyenne.gnu.log
View file
Open in desktop

Large diffs are not rendered by default.

2,115 changes: 1,305 additions & 810 deletions tests/RegressionTests_cheyenne.intel.log
View file
Open in desktop

Large diffs are not rendered by default.

2,103 changes: 1,299 additions & 804 deletions tests/RegressionTests_gaea.intel.log
View file
Open in desktop

Large diffs are not rendered by default.

835 changes: 590 additions & 245 deletions tests/RegressionTests_hera.gnu.log
View file
Open in desktop

Large diffs are not rendered by default.

2,123 changes: 1,309 additions & 814 deletions tests/RegressionTests_hera.intel.log
View file
Open in desktop

Large diffs are not rendered by default.

5,255 changes: 3,261 additions & 1,994 deletions tests/RegressionTests_jet.intel.log
View file
Open in desktop

Large diffs are not rendered by default.

2,125 changes: 1,310 additions & 815 deletions tests/RegressionTests_orion.intel.log
View file
Open in desktop

Large diffs are not rendered by default.

1,782 changes: 1,109 additions & 673 deletions tests/RegressionTests_wcoss2.intel.log
View file
Open in desktop

Large diffs are not rendered by default.

1,546 changes: 966 additions & 580 deletions tests/RegressionTests_wcoss_cray.log
View file
Open in desktop

Large diffs are not rendered by default.

2,211 changes: 1,361 additions & 850 deletions tests/RegressionTests_wcoss_dell_p3.log
View file
Open in desktop

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion tests/ci/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
From noaaemc/ubuntu-hpc:v1.10b
From noaaemc/ubuntu-hpc:v1.11b

ENV HOME=/home/builder
COPY --chown=builder:builder . $HOME/ufs-weather-model
Expand Down
4 changes: 2 additions & 2 deletions tests/ci/ci.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,13 +75,13 @@ if [ $BUILD = "true" ]; then
elif [ $RUN == "true" ]; then

docker volume rm -f DataVolume >/dev/null &&
docker run -d --rm -v DataVolume:/tmp minsukjinoaa/input-data:20220414
docker run -d --rm -v DataVolume:/tmp noaaemc/input-data:20220414

docker create -u builder -e "CI_TEST=true" -e "USER=builder" \
-e "RT_MACHINE=linux.gnu" -e "RT_COMPILER=gnu" \
-w "/home/builder/ufs-weather-model/tests" \
-v DataVolume:/home/builder/data/NEMSfv3gfs \
--shm-size=512m --name my-container noaaemc/ubuntu-hpc:v1.10b \
--shm-size=512m --name my-container noaaemc/ubuntu-hpc:v1.11b \
/bin/bash -c "./opnReqTest -n ${TEST_NAME} -c ${TEST_CASE} -x"

cd $GITHUB_WORKSPACE
Expand Down
2 changes: 1 addition & 1 deletion tests/ci/ci.test
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,5 @@ control
thr mpi dcp rst bit dbg
regional_control
thr dcp
cpld_control_noaero_p8
cpld_control_nowave_noaero_p8
rst dbg
4 changes: 3 additions & 1 deletion tests/fv3_conf/cpld_control_run.IN
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,9 @@ else
ls -1 ./INPUT/${RFILE}>ice.restart_file

# WAVE restart file
cp ../${DEP_RUN}${SUFFIX}/${RESTART_FILE_PREFIX}.restart.ww3 ./restart.ww3
if [[ $CPLWAV == .true. ]]; then
cp ../${DEP_RUN}${SUFFIX}/${RESTART_FILE_PREFIX}.restart.ww3 ./restart.ww3
fi
fi

#inline post
Expand Down
1 change: 1 addition & 0 deletions tests/opnReqTests/dbg.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ if [[ $application == 'global' ]]; then
else
LIST_FILES="sfcf0$FHMAX_2D.nc sfcf0$FHMAX_2D.nc"
fi
WRITE_DOPOST=.false.
elif [[ $application == 'regional' ]]; then
echo "Regional application not yet implemented for debug"
exit 1
Expand Down
1 change: 1 addition & 0 deletions tests/opnReqTests/wrt_env.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,4 +27,5 @@ export NFHOUT_HF=${NFHOUT_HF}
export LIST_FILES="${LIST_FILES}"
export OUTPUT_FH="${OUTPUT_FH}"
export AOD_FRQ=${AOD_FRQ:-}
export WRITE_DOPOST=${WRITE_DOPOST:-}
EOF
112 changes: 112 additions & 0 deletions tests/parm/nems.configure.cpld_noaero_nowave.IN
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
#############################################
#### NEMS Run-Time Configuration File #####
#############################################

# ESMF #
logKindFlag: ESMF_LOGKIND_MULTI

# EARTH #
EARTH_component_list: MED ATM OCN ICE
EARTH_attributes::
Verbosity = 0
::

# MED #
MED_model: @[med_model]
MED_petlist_bounds: @[med_petlist_bounds]
::

# ATM #
ATM_model: @[atm_model]
ATM_petlist_bounds: @[atm_petlist_bounds]
ATM_attributes::
Verbosity = 0
DumpFields = false
ProfileMemory = false
OverwriteSlice = true
::

# OCN #
OCN_model: @[ocn_model]
OCN_petlist_bounds: @[ocn_petlist_bounds]
OCN_attributes::
Verbosity = 0
DumpFields = false
ProfileMemory = false
OverwriteSlice = true
mesh_ocn = @[MESHOCN_ICE]
::

# ICE #
ICE_model: @[ice_model]
ICE_petlist_bounds: @[ice_petlist_bounds]
ICE_attributes::
Verbosity = 0
DumpFields = false
ProfileMemory = false
OverwriteSlice = true
mesh_ice = @[MESHOCN_ICE]
stop_n = @[RESTART_N]
stop_option = nhours
stop_ymd = -999
::

# CMEPS warm run sequence
runSeq::
@@[coupling_interval_slow_sec]
MED med_phases_prep_ocn_avg
MED -> OCN :remapMethod=redist
OCN
@@[coupling_interval_fast_sec]
MED med_phases_prep_atm
MED med_phases_prep_ice
MED -> ATM :remapMethod=redist
MED -> ICE :remapMethod=redist
ATM
ICE
ATM -> MED :remapMethod=redist
MED med_phases_post_atm
ICE -> MED :remapMethod=redist
MED med_phases_post_ice
MED med_phases_prep_ocn_accum
@
OCN -> MED :remapMethod=redist
MED med_phases_post_ocn
MED med_phases_restart_write
@
::

# CMEPS variables

DRIVER_attributes::
::

MED_attributes::
ATM_model = @[atm_model]
ICE_model = @[ice_model]
OCN_model = @[ocn_model]
history_n = 1
history_option = nhours
history_ymd = -999
coupling_mode = @[CPLMODE]
history_tile_atm = @[ATMTILESIZE]
::
ALLCOMP_attributes::
ScalarFieldCount = 2
ScalarFieldIdxGridNX = 1
ScalarFieldIdxGridNY = 2
ScalarFieldName = cpl_scalars
start_type = @[RUNTYPE]
restart_dir = RESTART/
case_name = ufs.cpld
restart_n = @[RESTART_N]
restart_option = nhours
restart_ymd = -999
dbug_flag = @[cap_dbug_flag]
use_coldstart = @[use_coldstart]
use_mommesh = @[use_mommesh]
eps_imesh = @[eps_imesh]
stop_n = @[FHMAX]
stop_option = nhours
stop_ymd = -999
::
2 changes: 1 addition & 1 deletion tests/parm/rrfs_conus13km_hrrr.nml.IN
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -234,7 +234,7 @@
! lfnc_k = @[LFNC_K]
! lfnc_p0 = @[LFNC_P0]
lheatstrg = @[LHEATSTRG]
! lkm = @[LKM]
lkm = @[LKM]
lndp_type = @[LNDP_TYPE]
lradar = @[LRADAR]
lrefres = .true.
Expand Down
28 changes: 19 additions & 9 deletions tests/rt.conf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# S2S tests #
###################################################################################################################################################################################

COMPILE | -DAPP=S2SWA -DCCPP_SUITES=FV3_GFS_v17_coupled_p8 | - wcoss_cray wcoss2.intel | fv3 |
COMPILE | -DAPP=S2SWA -DCCPP_SUITES=FV3_GFS_v17_coupled_p8,FV3_GFS_cpld_rasmgshocnsstnoahmp_ugwp | - wcoss_cray wcoss2.intel | fv3 |
RUN | cpld_control_p8 | - wcoss_cray wcoss2.intel | fv3 |
RUN | cpld_restart_p8 | - wcoss_cray wcoss2.intel | | cpld_control_p8
RUN | cpld_2threads_p8 | - wcoss_cray wcoss2.intel | |
Expand All @@ -16,9 +16,16 @@ RUN | cpld_restart_c192_p8
RUN | cpld_bmark_p8 | - wcoss_cray wcoss2.intel jet.intel | fv3 |
RUN | cpld_restart_bmark_p8 | - wcoss_cray wcoss2.intel jet.intel | | cpld_bmark_p8

COMPILE | -DAPP=S2SWA -DDEBUG=ON -DCCPP_SUITES=FV3_GFS_v17_coupled_p8 | - wcoss_cray wcoss2.intel | fv3 |
COMPILE | -DAPP=S2SW -DCCPP_SUITES=FV3_GFS_v17_coupled_p8 | | fv3 |
RUN | cpld_control_noaero_p8 | | fv3 |
RUN | cpld_control_nowave_noaero_p8 | | fv3 |

COMPILE | -DAPP=S2SWA -DDEBUG=ON -DCCPP_SUITES=FV3_GFS_v17_coupled_p8,FV3_GFS_cpld_rasmgshocnsstnoahmp_ugwp | - wcoss_cray wcoss2.intel | fv3 |
RUN | cpld_debug_p8 | - wcoss_cray wcoss2.intel | fv3 |

COMPILE | -DAPP=S2SW -DDEBUG=ON -DCCPP_SUITES=FV3_GFS_v17_coupled_p8 | | fv3 |
RUN | cpld_debug_noaero_p8 | - wcoss_cray wcoss2.intel | fv3 |

# Waves and aerosol off for computing fluxes in mediator
COMPILE | -DAPP=S2S -DCCPP_SUITES=FV3_GFS_v17_coupled_p8_sfcocn -DCMEPS_AOFLUX=ON | - wcoss_cray | fv3 |
RUN | cpld_control_noaero_p8_agrid | - wcoss_cray | fv3 |
Expand Down Expand Up @@ -60,24 +67,26 @@ RUN | regional_control
RUN | regional_restart | | | regional_control
RUN | regional_control_2dwrtdecomp | | |
RUN | regional_noquilt | | fv3 |
RUN | regional_2threads | - wcoss_cray jet.intel | |
RUN | regional_2threads | - wcoss_cray | |
RUN | regional_netcdf_parallel | | fv3 |
RUN | regional_3km | | fv3 |
RUN | regional_3km_decomp | | |

COMPILE | -DAPP=ATM -DCCPP_SUITES=FV3_RAP,FV3_RAP_RRTMGP,FV3_RAP_sfcdiff,FV3_HRRR,FV3_HRRR_smoke,FV3_RRFS_v1beta,FV3_RRFS_v1nssl -D32BIT=ON | | fv3 |

RUN | rap_control | | fv3 |
RUN | rap_rrtmgp | | fv3 |
RUN | regional_spp_sppt_shum_skeb | | fv3 |
# This test is not giving the same results (also not in REPRO mode) - needs to be fixed
#RUN | rap_decomp | | |
RUN | rap_2threads | - wcoss_cray jet.intel | |
RUN | rap_decomp | | |
RUN | rap_2threads | - wcoss_cray | |
RUN | rap_restart | | | rap_control
RUN | rap_sfcdiff | | fv3 |
# This test is not giving the same results (also not in REPRO mode) - needs to be fixed
#RUN | rap_sfcdiff_decomp | | |
RUN | rap_sfcdiff_decomp | | |
RUN | rap_sfcdiff_restart | | | rap_sfcdiff
RUN | hrrr_control | | fv3 |
RUN | hrrr_control_decomp | | |
RUN | hrrr_control_2threads | - wcoss_cray | |
RUN | hrrr_control_restart | | | hrrr_control
RUN | rrfs_v1beta | | fv3 |
RUN | rrfs_v1nssl | | fv3 |
RUN | rrfs_v1nssl_nohailnoccn | | fv3 |
Expand Down Expand Up @@ -124,9 +133,10 @@ RUN | control_thompson_progcld_thompson_debug

RUN | regional_debug | | fv3 |

COMPILE | -DAPP=ATM -DDEBUG=ON -DCCPP_SUITES=FV3_RAP,FV3_RAP_cires_ugwp,FV3_RAP_unified_ugwp,FV3_RAP_flake -D32BIT=ON | | fv3 |
COMPILE | -DAPP=ATM -DDEBUG=ON -DCCPP_SUITES=FV3_RAP,FV3_RAP_cires_ugwp,FV3_RAP_unified_ugwp,FV3_RAP_flake,FV3_HRRR -D32BIT=ON | | fv3 |

RUN | rap_control_debug | | fv3 |
RUN | hrrr_control_debug | | fv3 |
RUN | rap_unified_drag_suite_debug | | |
RUN | rap_diag_debug | | fv3 |
RUN | rap_cires_ugwp_debug | | fv3 |
Expand Down
2 changes: 1 addition & 1 deletion tests/rt.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -485,7 +485,7 @@ if [[ $TESTS_FILE =~ '35d' ]] || [[ $TESTS_FILE =~ 'weekly' ]]; then
TEST_35D=true
fi

BL_DATE=20220613
BL_DATE=20220616
if [[ $MACHINE_ID = hera.* ]] || [[ $MACHINE_ID = orion.* ]] || [[ $MACHINE_ID = cheyenne.* ]] || [[ $MACHINE_ID = gaea.* ]] || [[ $MACHINE_ID = jet.* ]] || [[ $MACHINE_ID = s4.* ]] || [[ $MACHINE_ID = wcoss2.* ]]; then
RTPWD=${RTPWD:-$DISKNM/NEMSfv3gfs/develop-${BL_DATE}/${RT_COMPILER^^}}
else
Expand Down
Loading