Skip to content

release/public-v2: b4b reproducibility for restart runs #417

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
junwang-noaa merged 10 commits into from
Feb 16, 2021
Merged

release/public-v2: b4b reproducibility for restart runs #417

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
4b042d8
Add coldstart and warmstart test for global fv3_ccpp_gfs_v15p2
climbfuji Feb 11, 2021
b28d218
coldstart and warmstart test for global fv3_ccpp_rrfs_v1alpha
climbfuji Feb 11, 2021
fe4cb79
Update .gitmodules and submodule pointer for fv3atm for code review a…
climbfuji Feb 11, 2021
70222c8
Add regional restart tests and update regression test baseline date tag
climbfuji Feb 12, 2021
45afb3e
Regression test log for orion.intel
climbfuji Feb 12, 2021
0ddae3e
Regression test logs for hera.intel, cheyenne.intel, cheyenne.gnu
climbfuji Feb 15, 2021
c6f95d7
Update submodule pointer for fv3atm
climbfuji Feb 16, 2021
aea570b
Regression test log for jet.intel
climbfuji Feb 16, 2021
e346a32
Regression test log for gaea.intel
climbfuji Feb 16, 2021
2c073e0
Revert change to .gitmodules and update submodule pointer for fv3atm
climbfuji Feb 16, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion FV3
Open in desktop
Submodule FV3 updated from 028e5d to bbd81f
313 changes: 250 additions & 63 deletions tests/RegressionTests_cheyenne.gnu.log
View file
Open in desktop

Large diffs are not rendered by default.

343 changes: 265 additions & 78 deletions tests/RegressionTests_cheyenne.intel.log
View file
Open in desktop

Large diffs are not rendered by default.

313 changes: 250 additions & 63 deletions tests/RegressionTests_gaea.intel.log
View file
Open in desktop

Large diffs are not rendered by default.

313 changes: 250 additions & 63 deletions tests/RegressionTests_hera.intel.log
View file
Open in desktop

Large diffs are not rendered by default.

313 changes: 250 additions & 63 deletions tests/RegressionTests_jet.intel.log
View file
Open in desktop

Large diffs are not rendered by default.

313 changes: 250 additions & 63 deletions tests/RegressionTests_orion.intel.log
View file
Open in desktop

Large diffs are not rendered by default.

11 changes: 7 additions & 4 deletions tests/fv3_conf/ccpp_gfs_v15_run.IN
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,14 @@
rm -fr INPUT RESTART
mkdir INPUT RESTART
if [ $WARM_START = .F. ]; then
cp -r @[RTPWD]/FV3_input_data/INPUT .
mkdir RESTART
cp -r @[RTPWD]/FV3_input_data/INPUT/* INPUT/
else
mkdir INPUT RESTART
cp -r @[RTPWD]/FV3_input_data/RESTART/* ./INPUT
cp -r @[RTPWD]/FV3_input_data/INPUT/grid_spec*.nc INPUT/
cp -r @[RTPWD]/FV3_input_data/INPUT/C96_grid*.nc INPUT/
cp -r @[RTPWD]/FV3_input_data/INPUT/oro_data*.nc INPUT/
cp ../fv3_ccpp_gfs_v15p2_coldstart${RT_SUFFIX}/RESTART/* INPUT/
fi

cp @[RTPWD]/FV3_input_data/INPUT/aerosol.dat .
cp @[RTPWD]/FV3_input_data/INPUT/co2historicaldata_201*.txt .
cp @[RTPWD]/FV3_input_data/INPUT/sfc_emissivity_idx.txt .
Expand Down
15 changes: 12 additions & 3 deletions tests/fv3_conf/ccpp_regional_run.IN
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,18 @@ rm -fr INPUT RESTART
mkdir INPUT RESTART
rsync -av @[RTPWD]/FV3_input_data_regional/ ./
if [ $WARM_START = .T. ]; then
cp ../fv3_ccpp_regional_coldstart${RT_SUFFIX}/RESTART/* INPUT/
cp ../fv3_ccpp_regional_coldstart${RT_SUFFIX}/INPUT/gfs_bndy.tile7.*.nc INPUT/
cp ../fv3_ccpp_regional_coldstart${RT_SUFFIX}/INPUT/oro_data.tile7.halo4.nc INPUT/
if [ $CCPP_SUITE = FV3_RRFS_v1alpha ]; then
cp ../fv3_ccpp_regional_coldstart${RT_SUFFIX}/RESTART/* INPUT/
cp ../fv3_ccpp_regional_coldstart${RT_SUFFIX}/INPUT/gfs_bndy.tile7.*.nc INPUT/
cp ../fv3_ccpp_regional_coldstart${RT_SUFFIX}/INPUT/oro_data.tile7.halo4.nc INPUT/
elif [ $CCPP_SUITE = FV3_GFS_v15p2 ]; then
cp ../fv3_ccpp_regional_v15p2_coldstart${RT_SUFFIX}/RESTART/* INPUT/
Comment thread
junwang-noaa marked this conversation as resolved.
cp ../fv3_ccpp_regional_v15p2_coldstart${RT_SUFFIX}/INPUT/gfs_bndy.tile7.*.nc INPUT/
cp ../fv3_ccpp_regional_v15p2_coldstart${RT_SUFFIX}/INPUT/oro_data.tile7.halo4.nc INPUT/
else
echo "ERROR, warm start runs not configured for CCPP_SUITE=${CCPP_SUITE}"
exit 1
fi
fi

# Copy field table, depending on microphysics choice
Expand Down
10 changes: 6 additions & 4 deletions tests/rt.conf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,21 +9,23 @@ RUN | fv3_ccpp_regional_control
RUN | fv3_ccpp_regional_decomp | standard | | |
RUN | fv3_ccpp_regional_2threads | standard | | |
RUN | fv3_ccpp_regional_coldstart | standard | | |
# The following test does NOT reproduce the control baseline
#RUN | fv3_ccpp_regional_warmstart | standard | | | fv3_ccpp_regional_coldstart
RUN | fv3_ccpp_regional_warmstart | standard | | | fv3_ccpp_regional_coldstart
RUN | fv3_ccpp_regional_v15p2 | standard | | fv3 |
RUN | fv3_ccpp_regional_v15p2_2threads | standard | | |
# This test only reproduces in REPRO mode (because of GFDL-MP fast physics in 32-bit)
#RUN | fv3_ccpp_regional_v15p2_decomp | standard | | |
RUN | fv3_ccpp_regional_v15p2_coldstart | standard | | |
RUN | fv3_ccpp_regional_v15p2_warmstart | standard | | | fv3_ccpp_regional_coldstart

# Global tests
RUN | fv3_ccpp_rrfs_v1alpha | standard | | fv3 |
RUN | fv3_ccpp_rrfs_v1alpha_decomp | standard | | |
RUN | fv3_ccpp_rrfs_v1alpha_2threads | standard | | |
RUN | fv3_ccpp_rrfs_v1alpha_coldstart | standard | | |
# The following test does NOT reproduce the control baseline
#RUN | fv3_ccpp_rrfs_v1alpha_warmstart | standard | | | fv3_ccpp_rrfs_v1alpha_coldstart
RUN | fv3_ccpp_rrfs_v1alpha_warmstart | standard | | | fv3_ccpp_rrfs_v1alpha_coldstart
RUN | fv3_ccpp_gfs_v15p2 | standard | | fv3 |
RUN | fv3_ccpp_gfs_v15p2_coldstart | standard | | |
RUN | fv3_ccpp_gfs_v15p2_warmstart | standard | | | fv3_ccpp_gfs_v15p2_coldstart

###############################################################################################################################
# CCPP REPRO TESTS #
Expand Down
4 changes: 2 additions & 2 deletions tests/rt.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -413,9 +413,9 @@ if [[ $SINGLE_NAME != '' ]]; then
fi

if [[ $MACHINE_ID = hera.* ]] || [[ $MACHINE_ID = orion.* ]] || [[ $MACHINE_ID = cheyenne.* ]]; then
RTPWD=${RTPWD:-$DISKNM/NEMSfv3gfs/ufs-public-release-v2-20210208/${COMPILER^^}}
RTPWD=${RTPWD:-$DISKNM/NEMSfv3gfs/ufs-public-release-v2-20210212/${COMPILER^^}}
else
RTPWD=${RTPWD:-$DISKNM/NEMSfv3gfs/ufs-public-release-v2-20210208}
RTPWD=${RTPWD:-$DISKNM/NEMSfv3gfs/ufs-public-release-v2-20210212}
fi

shift $((OPTIND-1))
Expand Down
14 changes: 14 additions & 0 deletions tests/tests/fv3_ccpp_gfs_v15p2
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,12 @@ export LIST_FILES="atmos_4xdaily.tile1.nc \
phyf000.tile4.nc \
phyf000.tile5.nc \
phyf000.tile6.nc \
phyf012.tile1.nc \
phyf012.tile2.nc \
phyf012.tile3.nc \
phyf012.tile4.nc \
phyf012.tile5.nc \
phyf012.tile6.nc \
phyf024.tile1.nc \
phyf024.tile2.nc \
phyf024.tile3.nc \
Expand All @@ -32,6 +38,12 @@ export LIST_FILES="atmos_4xdaily.tile1.nc \
dynf000.tile4.nc \
dynf000.tile5.nc \
dynf000.tile6.nc \
dynf012.tile1.nc \
dynf012.tile2.nc \
dynf012.tile3.nc \
dynf012.tile4.nc \
dynf012.tile5.nc \
dynf012.tile6.nc \
dynf024.tile1.nc \
dynf024.tile2.nc \
dynf024.tile3.nc \
Expand Down Expand Up @@ -75,6 +87,8 @@ export LIST_FILES="atmos_4xdaily.tile1.nc \
export_fv3
export NODES=$(expr $TASKS / $TPN + 1)

export FHMAX=24

DT_ATMOS="1200"

# Ozone / stratospheric H2O
Expand Down
54 changes: 54 additions & 0 deletions tests/tests/fv3_ccpp_gfs_v15p2_coldstart
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
###############################################################################
#
# FV3 CCPP GFS v15.2 compiled with 32-bit dynamics test
#
###############################################################################

export TEST_DESCR="Compare FV3 32bit CCPP GFS v15.2 results with previous trunk version"

export CNTL_DIR=fv3_gfs_v15p2

export LIST_FILES="phyf000.tile1.nc \
phyf000.tile2.nc \
phyf000.tile3.nc \
phyf000.tile4.nc \
phyf000.tile5.nc \
phyf000.tile6.nc \
phyf012.tile1.nc \
phyf012.tile2.nc \
phyf012.tile3.nc \
phyf012.tile4.nc \
phyf012.tile5.nc \
phyf012.tile6.nc \
dynf000.tile1.nc \
dynf000.tile2.nc \
dynf000.tile3.nc \
dynf000.tile4.nc \
dynf000.tile5.nc \
dynf000.tile6.nc \
dynf012.tile1.nc \
dynf012.tile2.nc \
dynf012.tile3.nc \
dynf012.tile4.nc \
dynf012.tile5.nc \
dynf012.tile6.nc"


export_fv3
export NODES=$(expr $TASKS / $TPN + 1)

export FHMAX=12

DT_ATMOS="1200"

# Ozone / stratospheric H2O
export OZ_PHYS_OLD=.F.
export OZ_PHYS_NEW=.T.
export H2O_PHYS=.T.

export FV3_RUN=ccpp_gfs_v15_run.IN
export CCPP_SUITE=FV3_GFS_v15p2
export CCPP_LIB_DIR=ccpp/lib
export INPUT_NML=ccpp_global.nml.IN

RUN_SCRIPT=rt_fv3.sh
82 changes: 82 additions & 0 deletions tests/tests/fv3_ccpp_gfs_v15p2_warmstart
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
###############################################################################
#
# FV3 CCPP GFS v15.2 compiled with 32-bit dynamics test
#
###############################################################################

export TEST_DESCR="Compare FV3 32bit CCPP GFS v15.2 results with previous trunk version"

export CNTL_DIR=fv3_gfs_v15p2

export LIST_FILES="phyf024.tile1.nc \
phyf024.tile2.nc \
phyf024.tile3.nc \
phyf024.tile4.nc \
phyf024.tile5.nc \
phyf024.tile6.nc \
dynf024.tile1.nc \
dynf024.tile2.nc \
dynf024.tile3.nc \
dynf024.tile4.nc \
dynf024.tile5.nc \
dynf024.tile6.nc \
RESTART/coupler.res \
RESTART/fv_core.res.nc \
RESTART/fv_core.res.tile1.nc \
RESTART/fv_core.res.tile2.nc \
RESTART/fv_core.res.tile3.nc \
RESTART/fv_core.res.tile4.nc \
RESTART/fv_core.res.tile5.nc \
RESTART/fv_core.res.tile6.nc \
RESTART/fv_srf_wnd.res.tile1.nc \
RESTART/fv_srf_wnd.res.tile2.nc \
RESTART/fv_srf_wnd.res.tile3.nc \
RESTART/fv_srf_wnd.res.tile4.nc \
RESTART/fv_srf_wnd.res.tile5.nc \
RESTART/fv_srf_wnd.res.tile6.nc \
RESTART/fv_tracer.res.tile1.nc \
RESTART/fv_tracer.res.tile2.nc \
RESTART/fv_tracer.res.tile3.nc \
RESTART/fv_tracer.res.tile4.nc \
RESTART/fv_tracer.res.tile5.nc \
RESTART/fv_tracer.res.tile6.nc \
RESTART/sfc_data.tile1.nc \
RESTART/sfc_data.tile2.nc \
RESTART/sfc_data.tile3.nc \
RESTART/sfc_data.tile4.nc \
RESTART/sfc_data.tile5.nc \
RESTART/sfc_data.tile6.nc \
RESTART/phy_data.tile1.nc \
RESTART/phy_data.tile2.nc \
RESTART/phy_data.tile3.nc \
RESTART/phy_data.tile4.nc \
RESTART/phy_data.tile5.nc \
RESTART/phy_data.tile6.nc"


export_fv3
export NODES=$(expr $TASKS / $TPN + 1)

export WARM_START=.T.
export NGGPS_IC=.F.
export EXTERNAL_IC=.F.
export MAKE_NH=.F.
export MOUNTAIN=.T.
export NA_INIT=0
export NSTF_NAME=2,0,1,0,5

export FHMAX=24

DT_ATMOS="1200"

# Ozone / stratospheric H2O
export OZ_PHYS_OLD=.F.
export OZ_PHYS_NEW=.T.
export H2O_PHYS=.T.

export FV3_RUN=ccpp_gfs_v15_run.IN
export CCPP_SUITE=FV3_GFS_v15p2
export CCPP_LIB_DIR=ccpp/lib
export INPUT_NML=ccpp_global.nml.IN

RUN_SCRIPT=rt_fv3.sh
2 changes: 1 addition & 1 deletion tests/tests/fv3_ccpp_regional_coldstart
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,4 +63,4 @@ export WRITE_RESTART_WITH_BCS=.true.

if [[ $MACHINE_ID = *.gnu ]]; then
export WLCLK=60
fi
fi
2 changes: 2 additions & 0 deletions tests/tests/fv3_ccpp_regional_v15p2
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,10 @@ export CNTL_DIR=fv3_regional_v15p2

export LIST_FILES="atmos_4xdaily.nc \
phyf000.nc \
phyf006.nc \
phyf012.nc \
dynf000.nc \
dynf006.nc \
dynf012.nc \
RESTART/coupler.res \
RESTART/fv_core.res.nc \
Expand Down
53 changes: 53 additions & 0 deletions tests/tests/fv3_ccpp_regional_v15p2_coldstart
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
###############################################################################
#
# FV3 CCPP regional control test using GFS v15p2 suite with 32-bit dynamics
#
###############################################################################

export TEST_DESCR="Compare FV3 CCPP regional GFS v15p2 results with previous trunk version"

export CNTL_DIR=fv3_regional_v15p2

export LIST_FILES="phyf000.nc \
phyf006.nc \
dynf000.nc \
dynf006.nc"

export_fv3
export TASKS=80
export NODES=$(expr $TASKS / $TPN + 1)

export INPES=20
export JNPES=2

export FHMAX=6
export FDIAG=3

DT_ATMOS="300"

export FV3_RUN=ccpp_regional_run.IN
export CCPP_SUITE=FV3_GFS_v15p2
export CCPP_LIB_DIR=ccpp/lib
export INPUT_NML=ccpp_regional.nml.IN
export MODEL_CONFIGURE=ccpp_regional-model_configure.IN

# Ozone / stratospheric H2O
export OZ_PHYS_OLD=.F.
export OZ_PHYS_NEW=.T.
export H2O_PHYS=.T.

# GFDL microphysics
export IMP_PHYSICS=11

# Other namelist switches
export HYBEDMF=.T.
export FHCYC=0

RUN_SCRIPT=rt_fv3.sh

# Regional option
export WRITE_RESTART_WITH_BCS=.true.

if [[ $MACHINE_ID = *.gnu ]]; then
export WLCLK=60
fi
66 changes: 66 additions & 0 deletions tests/tests/fv3_ccpp_regional_v15p2_warmstart
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
###############################################################################
#
# FV3 CCPP regional control test using GFS v15p2 suite with 32-bit dynamics
#
###############################################################################

export TEST_DESCR="Compare FV3 CCPP regional GFS v15p2 results with previous trunk version"

export CNTL_DIR=fv3_regional_v15p2

export LIST_FILES="phyf012.nc \
dynf012.nc \
RESTART/coupler.res \
RESTART/fv_core.res.nc \
RESTART/fv_core.res.tile1.nc \
RESTART/fv_srf_wnd.res.tile1.nc \
RESTART/fv_tracer.res.tile1.nc \
RESTART/sfc_data.nc \
RESTART/phy_data.nc"

export_fv3
export TASKS=80
export NODES=$(expr $TASKS / $TPN + 1)

export INPES=20
export JNPES=2

export WARM_START=.T.
export NGGPS_IC=.F.
export EXTERNAL_IC=.F.
export MAKE_NH=.F.
export MOUNTAIN=.T.
export NA_INIT=0
export NSTF_NAME=2,0,1,0,5

export FHMAX=12
export FDIAG=3

DT_ATMOS="300"

export FV3_RUN=ccpp_regional_run.IN
export CCPP_SUITE=FV3_GFS_v15p2
export CCPP_LIB_DIR=ccpp/lib
export INPUT_NML=ccpp_regional.nml.IN
export MODEL_CONFIGURE=ccpp_regional-model_configure.IN

# Ozone / stratospheric H2O
export OZ_PHYS_OLD=.F.
export OZ_PHYS_NEW=.T.
export H2O_PHYS=.T.

# GFDL microphysics
export IMP_PHYSICS=11

# Other namelist switches
export HYBEDMF=.T.
export FHCYC=0

RUN_SCRIPT=rt_fv3.sh

# Regional option
export WRITE_RESTART_WITH_BCS=.true.

if [[ $MACHINE_ID = *.gnu ]]; then
export WLCLK=60
fi
Loading