Faster compile and HAFS app to be compiled with 64bit

[sadeghitabas]

…MAQ in rt.conf

Description

This PR contains changes to rt.conf for FASTER compile UFS regression tests.
Changes to rt.conf:

• FASTER compile option turned on for ATM, S2SWA, NG-GODAS, ATML and ATMAQ

Top of commit queue on: TBD

Input data additions/changes

• No changes are expected to input data.
• There will be new input data.
• Input data will be updated.

Anticipated changes to regression tests:

• No changes are expected to any regression test.
• Changes are expected to the following tests:

Subcomponents involved:

• AQM
• CDEPS
• CICE
• CMEPS
• CMakeModules
• FV3
• GOCART
• HYCOM
• MOM6
• NOAHMP
• WW3
• stochastic_physics
• none

Combined with PR's (If Applicable):

PR #1610

Commit Queue Checklist:

• Link PR's from all sub-components involved
• Confirm reviews completed in sub-component PR's
• Add all appropriate labels to this PR.
• Run full RT suite on either Hera/Cheyenne with both Intel/GNU compilers
• Add list of any failed regression tests to "Anticipated changes to regression tests" section.

Linked PR's and Issues:

• Closes Test the FASTER compile option in UFS applications #1572
• Linked PR: Allow HAFS app to be compiled with 64bit FMS #1610

Testing Day Checklist:

• This PR is up-to-date with the top of all sub-component repositories except for those sub-components which are the subject of this PR.
• Move new/updated input data on RDHPCS Hera and propagate input data changes to all supported systems.

Testing Log (for CM's):

• RDHPCS
  • Intel
    • Hera
    • Orion
    • Jet
    • Gaea
    • Cheyenne
  • GNU
    • Hera
    • Cheyenne
• WCOSS2
  • Dogwood/Cactus
  • Acorn
• CI
  • Completed
• opnReqTest
  • N/A
  • Log attached to comment

[junwang-noaa]

@SadeghTabas-NOAA Code change looks good to me. Please run RT to check the run time change and if there are any failed test. Thanks

[github-actions]

@SadeghTabas-NOAA please bring these up to date with respective authoritative repositories

• ufs-weather-model NOT up to date
• cice NOT up to date

[sadeghitabas]

hera.intel job completed. RT passed, however the runtime increased significantly.

[sadeghitabas]

It looks like, the FASTER compile option makes the runs significantly slower than before. Please find the full RT results for Orion and Hera in the following directory:
Orion: /work/noaa/nems/stabas/git/PR1578/
Hera: /scratch1/NCEPDEV/nems/Sadegh.Tabas/git/SadeghTabas-NOAA/PR1578_20230130

[jkbk2004]

@SadeghTabas-NOAA I was wondering if it slows down across all cases? then, we may need more time for the pr.

[sadeghitabas]

@jkbk2004 I'm doing the RT test for jet and gaea as well, I'll let you know once its done, I'll upload the logs here.

[sadeghitabas]

RT job on jet completed, the FASTER compile option makes the runs significantly slower than before. Please find the full RT results for jet in the following directory:
jet: /lfs4/HFIP/h-nems/Sadegh.Tabas/git/SadeghTabas-NOAA/PR1578/

[BrianCurtis-NOAA]

Seems that the verdict is we should not use the FASTER option at all?

[sadeghitabas]

@BrianCurtis-NOAA I guess the current version of Faster Compile makes the process much slower and it needs modification or not using it at all.

[sadeghitabas]

I made a google sheet for the Faster Compile option results. It seems Faster Compile made the Compiles process slower, however it increased the speed for the Tests.

[jkbk2004]

@SadeghTabas-NOAA #1604 was merged. can you sync up? mainly log files. it might be easier way to save your log files and attach in the PR after sync up.

[jkbk2004]

@SadeghTabas-NOAA mom6/fv3 restart file format was updated. you can take a look: https://github.com/SadeghTabas-NOAA/ufs-weather-model/blob/FASTER_Compile_Test/tests/tests/control_p8 Another thing to check is this pr adds 4 *_faster test cases and no impact on all other cases. Then, we might combine with #1610. We need to point to @DusanJovic-NOAA hafs_64bit_fms fv3 branch. Sorry I am flipping my mind about combing PRs. If you agree, we can save time in that way.

[jkbk2004]

Automated RT Failure Notification
Machine: gaea
Compiler: intel
Job: RT
[RT] Repo location: /lustre/f2/pdata/ncep/emc.nemspara/autort/pr/1213491813/20230304203007/ufs-weather-model
[RT] Error: Test cpld_control_p8_faster 021 failed in check_result failed
[RT] Error: Test cpld_control_p8_faster 021 failed in run_test failed
Please make changes and add the following label back: gaea-intel-RT

[jkbk2004]

Cheyenne and Gaea sporadically unstable. re-submission of jobs worked out ok for failed cases.

[jkbk2004]

@BrianCurtis-NOAA @SadeghTabas-NOAA once tests are confirmed on wcoss2, this pr is ready for merging.

[DeniseWorthen]

@SadeghTabas-NOAA Could you please edit the title so that we can easily see in the commit list that this also committed the change for HAFs in 64 Bit mode?

[sadeghitabas]

@BrianCurtis-NOAA Thanks,
@DeniseWorthen I modified the title, thanks.

[BrianCurtis-NOAA]

@jkbk2004 @SadeghTabas-NOAA All WCOSS2 related testing is completed. Apologies for the delay.

[jkbk2004]

@SadeghTabas-NOAA fv3 pr was merged. Can you update submodule pointer and revert change in gitmodules?

[jkbk2004]

@SadeghTabas-NOAA can you make sure about fv3 hash? correct one is NOAA-EMC/ufsatm@8418428