Skip to content

Add new tests for DATM-MOM6-CICE6 model #256

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
junwang-noaa merged 8 commits into from
Oct 30, 2020
Merged

Add new tests for DATM-MOM6-CICE6 model #256

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
601d915
Added the following files for datm-mom6-cice6 tests:
binli2337 Oct 27, 2020
6964772
Revised scripts in the tests directory.
binli2337 Oct 28, 2020
2274d2e
Removed blank space in some coupled test files.
binli2337 Oct 28, 2020
ffbf14d
Added log file from Orion.
binli2337 Oct 28, 2020
30d6c8c
Revised scripts.
binli2337 Oct 29, 2020
b9f1162
Revised test description.
binli2337 Oct 29, 2020
14badc2
Revised scripts and added log file from Hera.
binli2337 Oct 29, 2020
92b32c4
Added log file from Orion.
binli2337 Oct 30, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
190 changes: 116 additions & 74 deletions tests/RegressionTests_hera.intel.log
View file
Open in desktop

Large diffs are not rendered by default.

178 changes: 110 additions & 68 deletions tests/RegressionTests_orion.intel.log
View file
Open in desktop

Large diffs are not rendered by default.

101 changes: 100 additions & 1 deletion tests/default_vars.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -184,6 +184,9 @@ fi

export_fv3 ()
{
export FV3=true
export S2S=false
export DATM=false
export THRD=1
export WLCLK=$WLCLK_dflt
export INPES=$INPES_dflt
Expand Down Expand Up @@ -323,13 +326,14 @@ export IAU_DRYMASSFIXER=.false.
# Regional
export WRITE_RESTART_WITH_BCS=.false.

export S2S=false
export coupling_interval_fast_sec=0
}

export_cpl ()
{
export FV3=true
export S2S=true
export DATM=false

export DAYS="1"
export FHMAX="24"
Expand Down Expand Up @@ -449,3 +453,98 @@ export_35d_run ()
export CNTL_DIR=""
export LIST_FILES=""
}
export_datm ()
{
export FV3=false
export S2S=false
Comment thread
binli2337 marked this conversation as resolved.
export DATM=true
export DAYS=1
export FHMAX=24
export WLCLK=30
export THRD=1
export FHROT='0'

# atm/ocn/ice resolution
# GEFS
export DATM_SRC="GEFS"
export FILENAME_BASE='gefs.'
export IATM=1536
export JATM=768
export ATMRES='C96'
export OCNRES='100'
export ICERES='1.00'
export NX_GLB=360
export NY_GLB=320

# nems.configure
export NEMS_CONFIGURE="nems.configure.datm.IN"
export med_model="nems"
export atm_model="datm"
export ocn_model="mom6"
export ice_model="cice6"
export atm_petlist_bounds="0 15"
export med_petlist_bounds="16 111"
export ocn_petlist_bounds="112 231"
Copy link
Copy Markdown
Collaborator

@DeniseWorthen DeniseWorthen Oct 28, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is giving 120 PEs to the ocean at 1deg resolution and 48 PEs to the ice. We should set appropriate values for the default 1deg resolution and then change them in the 1/4deg test.

Copy link
Copy Markdown
Collaborator

@DeniseWorthen DeniseWorthen Oct 29, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will we have an issue and future PR to correctly assign the right PEs for 1 deg ocean and ice?

Copy link
Copy Markdown
Contributor Author

@binli2337 binli2337 Oct 29, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The model runs very quickly using these PEs for the 1 degree ocean and ice.

Copy link
Copy Markdown
Collaborator

@DeniseWorthen DeniseWorthen Oct 29, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm sure it does. But does it really seem reasonable to assign the same PES whether you're running 1/4 deg or 1deg?

Copy link
Copy Markdown
Collaborator

@junwang-noaa junwang-noaa Oct 30, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How many PEs are 1 degree MOM6 and CICE6 using in the coupled model? We do want to use fewer tasks for low resolution runs. Can we include this in the next DATM PR?

Copy link
Copy Markdown
Collaborator

@DeniseWorthen DeniseWorthen Oct 30, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The 1/4 deg set up in the coupled model is OCN on 30 PE and ICE on 12. When I set up the coupled model tasking, I simply reduced the 1/4 deg by 50% for the 1/2 deg and another 50% for the 1deg. So 120,60,30 for the ocean and 48,24,12 for the ice.

Phil is running even lower PE counts I believe.

export ice_petlist_bounds="232 255"
export TASKS=256
export TPN=40
export NPROC_ICE='24'

export ENS_NUM=1
export SYEAR='2011'
export SMONTH='10'
export SDAY='01'
export SHOUR='00'
export CDATE=${SYEAR}${SMONTH}${SDAY}${SHOUR}

export NFHOUT=6
export FDIAG=6
export DT_ATMOS='900'
export DT_DYNAM_MOM6='1800'
export DT_THERM_MOM6='3600'
export CPL_SLOW=${DT_THERM_MOM6}
export CPL_FAST=${DT_ATMOS}
export coupling_interval_slow_sec=${CPL_SLOW}
export coupling_interval_fast_sec=${CPL_FAST}

export RESTART_N=${FHMAX}
export CPLMODE='nems_orig_data'
export cap_dbug_flag="0"
export use_coldstart=".false."
export RUNTYPE='startup'

export INPUT_NML=input.mom6.nml.IN
export MODEL_CONFIGURE=datm_configure.IN
export FIELD_TABLE="field_table"

# MOM6 defaults; 1 degree
export MOM_INPUT=MOM_input_template_100
export MOM6_RESTART_SETTING='n'
export MOM6_RIVER_RUNOFF='False'
export FRUNOFF=""
export CHLCLIM='"seawifs_1998-2006_smoothed_2X.nc"'
# this must be set False for restart repro
export MOM6_REPRO_LA='False'
# since CPL_SLOW is set to DT_THERM, this should be always be false
export MOM6_THERMO_SPAN='False'
# no WW3
export MOM6_USE_WAVES='False'

# CICE6 defaults; 1 degree
export DT_CICE=${DT_ATMOS}
export MESHICE="mesh.mx${OCNRES}.nc"
export CICEGRID="grid_cice_NEMS_mx${OCNRES}.nc"
export CICEMASK="kmtu_cice_NEMS_mx${OCNRES}.nc"
export RUNID='unknown'
# set large; restart frequency now controlled by restart_n in nems.configure
export DUMPFREQ='d'
export DUMPFREQ_N=1000
export USE_RESTART_TIME='.false.'
export RESTART_EXT='.false'
# setting to true will allow Frazil FW and Salt to be
# included in fluxes sent to ocean
export FRAZIL_FWSALT='.true.'
# default to write CICE average history files
export CICE_HIST_AVG='.true.'
export BL_SUFFIX=""
Comment thread
binli2337 marked this conversation as resolved.
}
26 changes: 26 additions & 0 deletions tests/fv3_conf/cpld_datm_cfsr.IN
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
mkdir INPUT DATM_INPUT RESTART history MOM6_OUTPUT
Copy link
Copy Markdown
Collaborator

@DeniseWorthen DeniseWorthen Oct 28, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The cpld_datm_cfsr and cpd_datm_gefs essentially identical. It means we need to maintain two files instead of one. I think they should be combined by exporting the variable name of 'cfsr' or 'gefs' and resolution dependent variables in the actual test (datm_ocn_ice_mx100_cfsr etc). All 4 tests could then be made to call cpld_datm_control.IN.

Copy link
Copy Markdown
Contributor Author

@binli2337 binli2337 Oct 29, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the next PR, both debug and restart tests will be added and cpld_datm_control.IN will be used.

Copy link
Copy Markdown
Contributor Author

@binli2337 binli2337 Oct 29, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The GNU tests and baselines will be added in future PRs.


export DATM=true
# FV3 fixed input
export IATM=1760
export JATM=880
cp @[RTPWD]/DATM/cfsr.SCRIP.nc DATM_INPUT
ln -s @[RTPWD]/DATM/CFSR/201110/* DATM_INPUT

# MOM6 fixed input
cp @[RTPWD]/MOM6_FIX/@[OCNRES]/* ./INPUT
cp @[RTPWD]/MOM6_FIX_DATM/@[OCNRES]/* ./INPUT

# CICE fixed input
cp @[RTPWD]/CICE_FIX/@[OCNRES]/grid_cice_NEMS_mx@[OCNRES].nc .
cp @[RTPWD]/CICE_FIX/@[OCNRES]/kmtu_cice_NEMS_mx@[OCNRES].nc .
cp @[RTPWD]/CICE_FIX/@[OCNRES]/mesh.mx@[OCNRES].nc .


if [[ $OCNRES == '025' ]]; then
cp @[RTPWD]/MOM6_IC/@[OCNRES]/2011100100/MOM*.nc ./INPUT
cp @[RTPWD]/CICE_IC/@[OCNRES]/cice_model_@[ICERES].cpc*.nc cice_model.res.nc
else
cp @[RTPWD]/MOM6_IC/@[OCNRES]/2011100100/MOM6_IC_TS*.nc ./INPUT/MOM6_IC_TS.nc
cp @[RTPWD]/CICE_IC/@[OCNRES]/cice_model_@[ICERES].cpc*.nc cice_model.res.nc
fi
24 changes: 24 additions & 0 deletions tests/fv3_conf/cpld_datm_gefs.IN
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
mkdir INPUT DATM_INPUT RESTART history MOM6_OUTPUT

export DATM=true
# FV3 fixed input
cp @[RTPWD]/DATM/gefs.SCRIP.nc DATM_INPUT
ln -s @[RTPWD]/DATM/GEFS/201110/* DATM_INPUT

# MOM6 fixed input
cp @[RTPWD]/MOM6_FIX/@[OCNRES]/* ./INPUT
cp @[RTPWD]/MOM6_FIX_DATM/@[OCNRES]/* ./INPUT

# CICE fixed input
cp @[RTPWD]/CICE_FIX/@[OCNRES]/grid_cice_NEMS_mx@[OCNRES].nc .
cp @[RTPWD]/CICE_FIX/@[OCNRES]/kmtu_cice_NEMS_mx@[OCNRES].nc .
cp @[RTPWD]/CICE_FIX/@[OCNRES]/mesh.mx@[OCNRES].nc .


if [[ $OCNRES == '025' ]]; then
cp @[RTPWD]/MOM6_IC/@[OCNRES]/2011100100/MOM*.nc ./INPUT
cp @[RTPWD]/CICE_IC/@[OCNRES]/cice_model_@[ICERES].cpc*.nc cice_model.res.nc
else
cp @[RTPWD]/MOM6_IC/@[OCNRES]/2011100100/MOM6_IC_TS*.nc ./INPUT/MOM6_IC_TS.nc
cp @[RTPWD]/CICE_IC/@[OCNRES]/cice_model_@[ICERES].cpc*.nc cice_model.res.nc
fi
22 changes: 22 additions & 0 deletions tests/parm/datm_configure.IN
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
total_member: 1
print_esmf: .false.
PE_MEMBER01: @[TASKS]
start_year: @[SYEAR]
start_month: @[SMONTH]
start_day: @[SDAY]
start_hour: @[SHOUR]
start_minute: 0
start_second: 0
nhours_fcst: @[FHMAX]
RUN_CONTINUE: .false.
ENS_SPS: .false.

dt_atmos: @[DT_ATMOS]
atm_coupling_interval_sec: @[coupling_interval_fast_sec]

iatm: @[IATM]
jatm: @[JATM]

cdate0: @[CDATE]
nfhout: @[NFHOUT]
filename_base: @[FILENAME_BASE]
24 changes: 24 additions & 0 deletions tests/parm/datm_data_table.IN
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# this file will be read as a nems config file to set
# fields which might not be available with all forcing
# sources
mean_zonal_moment_flx_atm .true.
mean_merid_moment_flx_atm .true.
inst_height_lowest .true.
inst_temp_height_lowest .true.
inst_spec_humid_height_lowest .true.
inst_zonal_wind_height_lowest .true.
inst_merid_wind_height_lowest .true.
inst_pres_height_lowest .true.
mean_down_sw_flx .true.
mean_down_lw_flx .true.
mean_up_lw_flx .true.
mean_net_lw_flx .false.
mean_sensi_heat_flx .true.
mean_laten_heat_flx .true.
mean_down_sw_vis_dir_flx .true.
mean_down_sw_vis_dif_flx .true.
mean_down_sw_ir_dir_flx .true.
mean_down_sw_ir_dif_flx .true.
inst_pres_height_surface .true.
mean_prec_rate .true.
mean_fprec_rate .true.
15 changes: 15 additions & 0 deletions tests/parm/input.mom6.nml.IN
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
&fms_nml
clock_grain='ROUTINE'
clock_flags='NONE'
domains_stack_size = 5000000
stack_size =0
/
&MOM_input_nml
output_directory = 'MOM6_OUTPUT/',
input_filename = '@[MOM6_RESTART_SETTING]'
restart_input_dir = 'INPUT/',
restart_output_dir = 'RESTART/',
parameter_filename = 'INPUT/MOM_input',
'INPUT/MOM_override' /


106 changes: 106 additions & 0 deletions tests/parm/nems.configure.datm.IN
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
#############################################
#### NEMS Run-Time Configuration File #####
#############################################

# EARTH #
EARTH_component_list: MED ATM OCN ICE
EARTH_attributes::
Verbosity = 0
::

# MED #
MED_model: @[med_model]
MED_petlist_bounds: @[med_petlist_bounds]
::

# ATM #
ATM_model: @[atm_model]
ATM_petlist_bounds: @[atm_petlist_bounds]
ATM_attributes::
Verbosity = 0
DumpFields = false
ProfileMemory = false
OverwriteSlice = true
DebugFlag = 0
::

# OCN #
OCN_model: @[ocn_model]
OCN_petlist_bounds: @[ocn_petlist_bounds]
OCN_attributes::
Verbosity = 0
DumpFields = false
ProfileMemory = false
OverwriteSlice = true
::

# ICE #
ICE_model: @[ice_model]
ICE_petlist_bounds: @[ice_petlist_bounds]
ICE_attributes::
Verbosity = 0
DumpFields = false
ProfileMemory = false
OverwriteSlice = true
mesh_ice = @[MESHICE]
stop_n = @[RESTART_N]
stop_option = nhours
stop_ymd = -999
::

# CMEPS concurrent warm run sequence

runSeq::
@@[coupling_interval_slow_sec]
MED med_phases_prep_ocn_accum_avg
MED -> OCN :remapMethod=redist
OCN
@@[coupling_interval_fast_sec]
MED med_phases_prep_ice
MED -> ICE :remapMethod=redist
ATM
ICE
ATM -> MED :remapMethod=redist
ICE -> MED :remapMethod=redist
MED med_fraction_set
MED med_phases_prep_ocn_map
MED med_phases_aofluxes_run
MED med_phases_prep_ocn_merge
MED med_phases_prep_ocn_accum_fast
@
OCN -> MED :remapMethod=redist
MED med_phases_restart_write
@
::

# CMEPS variables

DRIVER_attributes::
mediator_read_restart = @[use_coldstart]
::
MED_attributes::
ATM_model = datm
ICE_model = cice6
OCN_model = mom
history_n = 0
history_option = nhours
history_ymd = -999
coupling_mode = nems_orig_data
::
ALLCOMP_attributes::
ScalarFieldCount = 2
ScalarFieldIdxGridNX = 1
ScalarFieldIdxGridNY = 2
ScalarFieldName = cpl_scalars
start_type = @[RUNTYPE]
case_name = RESTART/DATM_@[DATM_SRC]
restart_n = @[RESTART_N]
restart_option = nhours
restart_ymd = -999
dbug_flag = @[cap_dbug_flag]
use_coldstart = @[use_coldstart]
coldair_outbreak_mod = .false.
flds_wiso = .false.
flux_convergence = 0.0
flux_max_iteration = 1
::
6 changes: 5 additions & 1 deletion tests/rt.conf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -202,4 +202,8 @@ RUN | cpld_debug
# Data Atmosphere tests
#######################################################################################################################################################################################

COMPILE | DATM=Y S2S=Y | standard | hera.intel orion.intel wcoss_dell_p3 | fv3 |
COMPILE | DATM=Y S2S=Y | standard | hera.intel orion.intel | fv3 |
RUN | datm_control_cfsr | standard | hera.intel orion.intel wcoss_dell_p3 | fv3 |
RUN | datm_control_gefs | standard | hera.intel orion.intel wcoss_dell_p3 | fv3 |
RUN | datm_mx025_cfsr | standard | hera.intel orion.intel wcoss_dell_p3 | fv3 |
RUN | datm_mx025_gefs | standard | hera.intel orion.intel wcoss_dell_p3 | fv3 |
15 changes: 10 additions & 5 deletions tests/run_test.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,10 +79,12 @@ cp ${PATHRT}/modules.fv3_${COMPILE_NR} modules.fv3

# Get the shell file that loads the "module" command and purges modules:
cp ${PATHRT}/../NEMS/src/conf/module-setup.sh.inc module-setup.sh
cp ${PATHRT}/parm/post_itag itag
cp ${PATHRT}/parm/postxconfig-NT.txt postxconfig-NT.txt
cp ${PATHRT}/parm/postxconfig-NT_FH00.txt postxconfig-NT_FH00.txt
cp ${PATHRT}/parm/params_grib2_tbl_new params_grib2_tbl_new
if [[ $FV3 = 'true' ]]; then
cp ${PATHRT}/parm/post_itag itag
cp ${PATHRT}/parm/postxconfig-NT.txt postxconfig-NT.txt
cp ${PATHRT}/parm/postxconfig-NT_FH00.txt postxconfig-NT_FH00.txt
cp ${PATHRT}/parm/params_grib2_tbl_new params_grib2_tbl_new
fi

SRCD="${PATHTR}"
RUND="${RUNDIR}"
Expand All @@ -102,7 +104,7 @@ fi
# Set up the run directory
source ./fv3_run

if [[ $S2S = 'true' ]]; then
if [[ $DATM = 'true' ]] || [[ $S2S = 'true' ]]; then
edit_ice_in < ${PATHRT}/parm/ice_in_template > ice_in
edit_mom_input < ${PATHRT}/parm/${MOM_INPUT:-MOM_input_template_$OCNRES} > INPUT/MOM_input
edit_diag_table < ${PATHRT}/parm/diag_table_template > diag_table
Expand All @@ -112,6 +114,9 @@ if [[ $S2S = 'true' ]]; then
cp ${PATHRT}/parm/pio_in pio_in
cp ${PATHRT}/parm/med_modelio.nml med_modelio.nml
fi
if [[ $DATM = 'true' ]]; then
cp ${PATHRT}/parm/datm_data_table.IN datm_data_table
fi

if [[ $SCHEDULER = 'pbs' ]]; then
NODES=$(( TASKS / TPN ))
Expand Down
Loading