Updates to ice_in.IN for use in global applications

[aerorahul]

Commit Queue Requirements:

• Fill out all sections of this template.
• All sub component pull requests have been reviewed by their code managers.
• Run the full RT suite (compared to current baselines) on either Hera/Derecho/Hercules AND have committed the log to my PR branch.
• Add list of all failed regression tests in "Regression Tests" section.

PR Information

Description

• Resolves Allow use of ice_in.IN in global applications #2068

Commit Message

Updates to ice_in.IN for use in global applications

Priority

• Critical Bugfix (This PR contains a critical bug fix and should be prioritized.)
• High (This PR contains a feature or fix needed for a time-sensitive project (eg GFSv17))
• Normal

Blocking Dependencies

Git Issues Fixed By This PR

Fixes #2068

Changes

Subcomponent (with links)

• AQM
• CDEPS
• CICE
• CMEPS
• CMakeModules
• FV3
• GOCART
• HYCOM
• MOM6
• NOAHMP
• WW3
• stochastic_physics
• none

Input data

• No changes are expected to input data.
• Changes are expected to input data:
  • New input data.
  • Updated input data.

Regression Tests:

• No changes are expected to any regression test.
• Changes are expected to the following tests:

FAILED REGRESSION TESTS

Libraries

• Not Needed
• Needed
  • Create separate issue in JCSDA/spack-stack asking for update to library. Include library name, library version.
  • Add issue link from JCSDA/spack-stack following this item

Testing Log:

• RDHPCS
  • Hera
  • Orion
  • Hercules
  • Jet
  • Gaea
  • Derecho
• WCOSS2
  • Dogwood/Cactus
  • Acorn
• CI
  • Completed
• opnReqTest
  • N/A
  • Log attached to comment

[DeniseWorthen]

@aerorahul The debug tests using CICE failed on Hercules w/ this PR. I need to check what is going on

rt_006_cpld_debug_gfsv17_intel.log:Test 006 cpld_debug_gfsv17_intel FAIL
rt_022_cpld_debug_p8_intel.log:Test 022 cpld_debug_p8_intel FAIL
rt_023_cpld_debug_noaero_p8_intel.log:Test 023 cpld_debug_noaero_p8_intel FAIL
rt_030_cpld_debug_pdlib_p8_intel.log:Test 030 cpld_debug_pdlib_p8_intel FAIL
rt_163_datm_cdeps_debug_cfsr_intel.log:Test 163 datm_cdeps_debug_cfsr_intel FAIL
rt_230_cpld_control_p8_gnu.log:Test 230 cpld_control_p8_gnu FAIL

In err log:
179: [hercules-07-49:649002:0:649002] Caught signal 8 (Floating point exception: floating-point invalid operation)

[DeniseWorthen]

It is failing in ice_history.F90:

https://github.com/NOAA-EMC/CICE/blob/50aa2c97882fbc9d4918813a22169fe97b424564/cicecore/cicedyn/analysis/ice_history.F90#L4486

[DeniseWorthen]

@aerorahul It appears the change for the variables f_divu and f_shear actually requires the upcoming CICE PR, which included a fix made in Consortium PR CICE-Consortium/CICE#906

* ice_flux: zero-initialize divu and shear in init_history_dyn

'divu' and 'shear' are accessed in 'accum_hist' when writing the initial
condition before they are initialized at the start of {eap, evp,
implicit_solver}. This leads to runtime error when compiling with NaN
initialization.

Zero-initialize 'divu' and 'shear' in init_history_dyn, where the
related variable 'strength' is already zero-initialized.

I was able to run the debug datm test if I compiled w/ my CICE PR branch.

[aerorahul]

How does this work in the global-workflow and why does it fail only on Hercules?

[DeniseWorthen]

You're not running in debug mode, right? And I did test also on Hera and it failed in the same way.

On Hercules, only the debug tests (which use NaN initialization) failed.

EDIT: The cpld_control_p8_gnu test which failed on Hercules was a time-out.

[aerorahul]

correct. The global-workflow does not run in debug mode. A developer may choose to run in debug mode, but that is UAYOR

[DeniseWorthen]

Can you proceed w/ your G-W PR, and I will include the ice_in changes in my CICE PR?

[DeniseWorthen]

@junwang-noaa @jkbk2004 This was going to be a simple non-baseline change but it turns out it will require the CICE PR I have pending. Since it is a high priority for G-W, can we prioritize the CICE-PR and I will wrap this change in.

[BrianCurtis-NOAA]

Is there a G-W issue to implement these changes that can be linked here?

[DeniseWorthen]

NOAA-EMC/global-workflow#2051

[DeniseWorthen]

I've pulled these changes into #2027

[aerorahul]

Closing as the changes have been pulled into #2027