Skip to content

Make aerosol output frequency variable#2982

Merged
DavidHuber-NOAA merged 23 commits into
NOAA-EMC:developfrom
WalterKolczynski-NOAA:feature/gocart_variable_fh
Nov 22, 2024
Merged

Make aerosol output frequency variable#2982
DavidHuber-NOAA merged 23 commits into
NOAA-EMC:developfrom
WalterKolczynski-NOAA:feature/gocart_variable_fh

Conversation

@WalterKolczynski-NOAA
Copy link
Copy Markdown
Contributor

@WalterKolczynski-NOAA WalterKolczynski-NOAA commented Oct 4, 2024

Description

The AERO_HISTORY.rc file that controls the GOCART output is turned into a template usint @[ ] syntax to allow for variable output times. A new "parsing" script is created for GOCART that handles parsing the AERO_HISTORY.rc file through atparse. Other manip- ulation of these files is also moved to the new parsing script to be consistent with other components.

A new variable, FHOUT_AERO is introduced to control the output frequency of the GOCART output. This is initially set to 3 to match other components (this is a change from the current value of 6). Since some aerosol fields are also included as part of the atmos output, it may be desirable to meld this back into FHOUT later. However, the atmos output has two frequencies (FHOUT_HF and FHOUT), a feature not supported by other components.

The RUN lists for aero fcst, aero analysis, and waves is simplified to avoid confusion, since there is no need for them to only be set if the components are on.

Resolves #2072

Type of change

  • Bug fix (fixes something broken)
  • New feature (adds functionality)
  • Maintenance (code refactor, clean-up, new CI test, etc.)

Change characteristics

  • Is this a breaking change (a change in existing functionality)? NO
  • Does this change require a documentation update? NO
  • Does this change require an update to any of the following submodules? NO

How has this been tested?

  • S2SWA forecast-only on Hercules (manually modifying AERO_FCST_RUN)

Checklist

  • Any dependent changes have been merged and published
  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have documented my code, including function, input, and output descriptions
  • My changes generate no new warnings
  • New and existing tests pass with my changes
  • This change is covered by an existing CI test or a new one has been added
  • I have made corresponding changes to the system documentation if necessary

Comment thread ush/parsing_namelists_GOCART.sh Fixed
DavidHuber-NOAA
DavidHuber-NOAA previously approved these changes Oct 4, 2024
Comment thread parm/config/gfs/config.base
Comment thread ush/parsing_namelists_GOCART.sh
CoryMartin-NOAA
CoryMartin-NOAA previously approved these changes Oct 4, 2024
Copy link
Copy Markdown
Contributor

@CoryMartin-NOAA CoryMartin-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't test but this looks good to me

Comment thread parm/config/gfs/config.base
aerorahul
aerorahul previously approved these changes Oct 4, 2024
Copy link
Copy Markdown
Contributor

@aerorahul aerorahul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.
Question:
Not for this PR to address -- how diffferent is the parm/ufs/gocart/AERO_HISTORY.rc file in the global-workflow from the one in ufs-weather-model?

@WalterKolczynski-NOAA
Copy link
Copy Markdown
Contributor Author

WalterKolczynski-NOAA commented Oct 4, 2024

Looks good. Question: Not for this PR to address -- how diffferent is the parm/ufs/gocart/AERO_HISTORY.rc file in the global-workflow from the one in ufs-weather-model?

@aerorahul Looks like they have already templated the one frequency we use, but not the others. A couple other small changes (also whitespace changes not shown):

diff parm/ufs/gocart/AERO_HISTORY.rc sorc/ufs_model.fd/tests/parm/gocart/AERO_HISTORY.rc.IN -b
384c384
<   inst_aod.frequency:  060000,
---
>   inst_aod.frequency:  @[AOD_FRQ],
390a391
>                        'NIEXTTAU'      , 'NI'       , 'AOD_NI',
405,406c406,407
<   tavg_2d_rad.fields:     'CA.bcEXTTAU' , 'CA.bc' ,
< 			  'CA.ocEXTTAU' , 'CA.oc' ,
---
>   tavg_2d_rad.fields:     'CAEXTTAUCA.bc' , 'CA.bc' ,
>                           'CAEXTTAUCA.oc' , 'CA.oc' ,

(This is against our develop, not the changes in this PR.)

@WalterKolczynski-NOAA WalterKolczynski-NOAA added the CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera label Oct 7, 2024
@emcbot emcbot added CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera and removed CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera labels Oct 7, 2024
@TerrenceMcGuinness-NOAA TerrenceMcGuinness-NOAA removed the CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera label Oct 7, 2024
@WalterKolczynski-NOAA WalterKolczynski-NOAA added the CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera label Oct 7, 2024
@emcbot emcbot added CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress CI-Hera-Failed **Bot use only** CI testing on Hera for this PR has failed and removed CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress labels Oct 7, 2024
@emcbot
Copy link
Copy Markdown

emcbot commented Oct 8, 2024

Experiment C48_ATM FAILED on Hera in Build# 2 in
/scratch1/NCEPDEV/global/CI/2982/RUNTESTS/EXPDIR/C48_ATM_8a18d626

@emcbot
Copy link
Copy Markdown

emcbot commented Oct 8, 2024

Experiment C48mx500_3DVarAOWCDA FAILED on Hera in Build# 2 in
/scratch1/NCEPDEV/global/CI/2982/RUNTESTS/EXPDIR/C48mx500_3DVarAOWCDA_8a18d626

@emcbot
Copy link
Copy Markdown

emcbot commented Oct 8, 2024

Experiment C48_S2SWA_gefs FAILED on Hera in Build# 2 in
/scratch1/NCEPDEV/global/CI/2982/RUNTESTS/EXPDIR/C48_S2SWA_gefs_8a18d626

@emcbot
Copy link
Copy Markdown

emcbot commented Oct 8, 2024

Experiment C96_atm3DVar FAILED on Hera in Build# 2 in
/scratch1/NCEPDEV/global/CI/2982/RUNTESTS/EXPDIR/C96_atm3DVar_8a18d626

@emcbot
Copy link
Copy Markdown

emcbot commented Oct 8, 2024

Experiment C48_S2SW FAILED on Hera in Build# 2 in
/scratch1/NCEPDEV/global/CI/2982/RUNTESTS/EXPDIR/C48_S2SW_8a18d626

@emcbot
Copy link
Copy Markdown

emcbot commented Oct 8, 2024

Experiment C96C48_hybatmaerosnowDA FAILED on Hera in Build# 2 in
/scratch1/NCEPDEV/global/CI/2982/RUNTESTS/EXPDIR/C96C48_hybatmaerosnowDA_8a18d626

@emcbot
Copy link
Copy Markdown

emcbot commented Oct 8, 2024

Experiment C96C48_hybatmDA FAILED on Hera in Build# 2 in
/scratch1/NCEPDEV/global/CI/2982/RUNTESTS/EXPDIR/C96C48_hybatmDA_8a18d626

@emcbot
Copy link
Copy Markdown

emcbot commented Oct 8, 2024

Experiment C96C48_ufs_hybatmDA FAILED on Hera in Build# 2 in
/scratch1/NCEPDEV/global/CI/2982/RUNTESTS/EXPDIR/C96C48_ufs_hybatmDA_8a18d626

@emcbot emcbot added CI-Hera-Failed **Bot use only** CI testing on Hera for this PR has failed and removed CI-Hera-Failed **Bot use only** CI testing on Hera for this PR has failed labels Oct 8, 2024
@emcbot
Copy link
Copy Markdown

emcbot commented Nov 13, 2024

Experiment C96_S2SWA_gefs_replay_ics FAILED on Hercules in Build# 2 with error logs:

/work2/noaa/global/CI/HERCULES/2982/RUNTESTS/COMROOT/C96_S2SWA_gefs_replay_ics_2aca0553/logs/2020110100/gefs_fcst_mem001_seg0.log
/work2/noaa/global/CI/HERCULES/2982/RUNTESTS/COMROOT/C96_S2SWA_gefs_replay_ics_2aca0553/logs/2020110100/gefs_fcst_mem002_seg0.log

Follow link here to view the contents of the above file(s): (link)

@emcbot
Copy link
Copy Markdown

emcbot commented Nov 13, 2024

Experiment C96_S2SWA_gefs_replay_ics FAILED on Hercules in Build# 2 in
/work2/noaa/global/CI/HERCULES/2982/RUNTESTS/EXPDIR/C96_S2SWA_gefs_replay_ics_2aca0553

@emcbot
Copy link
Copy Markdown

emcbot commented Nov 13, 2024

CI Failed on Hercules in Build# 2
Built and ran in directory /work2/noaa/global/CI/HERCULES/2982


Experiment C48_ATM_2aca0553 Completed 2 Cycles: *SUCCESS* at Wed Nov 13 04:37:06 CST 2024
Experiment C96_S2SWA_gefs_replay_ics_2aca0553 Terminated with 0
FAIL
FAIL tasks failed and 2 dead at Wed Nov 13 05:01:30 CST 2024
Experiment C96_S2SWA_gefs_replay_ics_2aca0553 Terminated: *FAIL*
Error logs:
/work2/noaa/global/CI/HERCULES/2982/RUNTESTS/COMROOT/C96_S2SWA_gefs_replay_ics_2aca0553/logs/2020110100/gefs_fcst_mem001_seg0.log
/work2/noaa/global/CI/HERCULES/2982/RUNTESTS/COMROOT/C96_S2SWA_gefs_replay_ics_2aca0553/logs/2020110100/gefs_fcst_mem002_seg0.log
Experiment C96_atm3DVar_2aca0553 Completed 3 Cycles: *SUCCESS* at Wed Nov 13 06:26:47 CST 2024
Experiment C48_S2SWA_gefs_2aca0553 Completed 1 Cycles: *SUCCESS* at Wed Nov 13 06:45:52 CST 2024
Experiment C96C48_hybatmDA_2aca0553 Completed 3 Cycles: *SUCCESS* at Wed Nov 13 07:03:19 CST 2024
Experiment C48_S2SW_2aca0553 Completed 2 Cycles: *SUCCESS* at Wed Nov 13 07:33:58 CST 2024

@emcbot
Copy link
Copy Markdown

emcbot commented Nov 14, 2024

Build FAILED on Hercules in Build# 3 with error logs:

/work2/noaa/global/CI/HERCULES/2982/gefs/sorc/logs/build_upp.log

Follow link here to view the contents of the above file(s): (link)

Makes GOCART output times a function since it is needed in two places.
The defulats file for GEFS was missing an entry for aero to ensure
@field@ get parsed at expt setup time. This was added to gfs as
part of NOAA-EMC#3053, but gefs was missed.
@emcbot
Copy link
Copy Markdown

emcbot commented Nov 14, 2024

Experiment C96C48_hybatmaerosnowDA FAILED on Hera in Build# 4 with error logs:

/scratch1/NCEPDEV/global/CI/2982/RUNTESTS/COMROOT/C96C48_hybatmaerosnowDA_d953c49c/logs/2021122018/gdas_fcst_seg0.log

Follow link here to view the contents of the above file(s): (link)

@emcbot
Copy link
Copy Markdown

emcbot commented Nov 14, 2024

Experiment C96C48_hybatmaerosnowDA FAILED on Hera in Build# 4 in
/scratch1/NCEPDEV/global/CI/2982/RUNTESTS/EXPDIR/C96C48_hybatmaerosnowDA_d953c49c

@emcbot
Copy link
Copy Markdown

emcbot commented Nov 14, 2024

CI Failed on Hera in Build# 4
Built and ran in directory /scratch1/NCEPDEV/global/CI/2982


Experiment C96_S2SWA_gefs_replay_ics_d953c49c Completed 1 Cycles: *SUCCESS* at Thu Nov 14 21:06:44 UTC 2024
Experiment C48mx500_3DVarAOWCDA_d953c49c Completed 2 Cycles: *SUCCESS* at Thu Nov 14 21:24:56 UTC 2024
Experiment C48_ATM_d953c49c Completed 2 Cycles: *SUCCESS* at Thu Nov 14 21:24:56 UTC 2024
Experiment C96C48_hybatmaerosnowDA_d953c49c Terminated with 0
FAIL
FAIL tasks failed and 1 dead at Thu Nov 14 21:31:11 UTC 2024
Experiment C96C48_hybatmaerosnowDA_d953c49c Terminated: *FAIL*
Error logs:
/scratch1/NCEPDEV/global/CI/2982/RUNTESTS/COMROOT/C96C48_hybatmaerosnowDA_d953c49c/logs/2021122018/gdas_fcst_seg0.log
Experiment C48_S2SWA_gefs_d953c49c Completed 1 Cycles: *SUCCESS* at Thu Nov 14 22:38:52 UTC 2024
Experiment C96_atm3DVar_d953c49c Completed 3 Cycles: *SUCCESS* at Thu Nov 14 22:50:13 UTC 2024
Experiment C96C48_hybatmDA_d953c49c Completed 3 Cycles: *SUCCESS* at Thu Nov 14 22:50:33 UTC 2024
Experiment C48_S2SW_d953c49c Completed 2 Cycles: *SUCCESS* at Thu Nov 14 23:08:33 UTC 2024
Experiment C96C48_ufs_hybatmDA_d953c49c Completed 3 Cycles: *SUCCESS* at Thu Nov 14 23:08:48 UTC 2024

@emcbot
Copy link
Copy Markdown

emcbot commented Nov 19, 2024

Experiment C96C48_hybatmaerosnowDA FAILED on Hera in Build# 1 with error logs:

/scratch1/NCEPDEV/global/CI/2982/RUNTESTS/COMROOT/C96C48_hybatmaerosnowDA_90b49ff2/logs/2021122018/gfs_arch.log

Follow link here to view the contents of the above file(s): (link)

@emcbot
Copy link
Copy Markdown

emcbot commented Nov 19, 2024

Experiment C96C48_hybatmaerosnowDA FAILED on Hera in Build# 1 in
/scratch1/NCEPDEV/global/CI/2982/RUNTESTS/EXPDIR/C96C48_hybatmaerosnowDA_90b49ff2

Comment thread parm/archive/master_gfs.yaml.j2 Outdated
@emcbot
Copy link
Copy Markdown

emcbot commented Nov 19, 2024

Experiment C96C48_ufs_hybatmDA FAILED on Hera in Build# 1 in
/scratch1/NCEPDEV/global/CI/2982/RUNTESTS/EXPDIR/C96C48_ufs_hybatmDA_90b49ff2

@emcbot
Copy link
Copy Markdown

emcbot commented Nov 19, 2024

CI Failed on Hera in Build# 1
Built and ran in directory /scratch1/NCEPDEV/global/CI/2982


Experiment C96C48_ufs_hybatmDA_90b49ff2 Terminated with  tasks failed and  dead at Tue Nov 19 18:42:49 UTC 2024
Experiment C96C48_ufs_hybatmDA_90b49ff2 Terminated: **

@emcbot
Copy link
Copy Markdown

emcbot commented Nov 20, 2024

CI Passed on Hera in Build# 2
Built and ran in directory /scratch1/NCEPDEV/global/CI/2982


**CI Failed** on Hera in Build# 1<br>Built and ran in directory `/scratch1/NCEPDEV/global/CI/2982`

Experiment C96C48_ufs_hybatmDA_90b49ff2 Terminated with tasks failed and dead at Tue Nov 19 18:42:49 UTC 2024
Experiment C96C48_ufs_hybatmDA_90b49ff2 Terminated: **
Experiment C96_S2SWA_gefs_replay_ics_3070f0fd Completed 1 Cycles: SUCCESS at Tue Nov 19 23:41:09 UTC 2024
Experiment C48_ATM_3070f0fd Completed 2 Cycles: SUCCESS at Tue Nov 19 23:52:34 UTC 2024
Experiment C48mx500_3DVarAOWCDA_3070f0fd Completed 2 Cycles: SUCCESS at Wed Nov 20 00:10:50 UTC 2024
Experiment C48_S2SW_3070f0fd Completed 2 Cycles: SUCCESS at Wed Nov 20 00:35:13 UTC 2024
Experiment C48_S2SWA_gefs_3070f0fd Completed 1 Cycles: SUCCESS at Wed Nov 20 01:32:00 UTC 2024
Experiment C96_atm3DVar_3070f0fd Completed 3 Cycles: SUCCESS at Wed Nov 20 01:48:49 UTC 2024
Experiment C96C48_hybatmDA_3070f0fd Completed 3 Cycles: SUCCESS at Wed Nov 20 02:18:41 UTC 2024
Experiment C96C48_hybatmaerosnowDA_3070f0fd Completed 3 Cycles: SUCCESS at Wed Nov 20 02:24:46 UTC 2024
Experiment C96C48_ufs_hybatmDA_3070f0fd Completed 3 Cycles: SUCCESS at Wed Nov 20 03:26:17 UTC 2024

@WalterKolczynski-NOAA
Copy link
Copy Markdown
Contributor Author

CI Tests set up to run in /lfs/h2/emc/ptmp/walter.kolczynski/PR/PR_2982/RUNTESTS on WCOSS

@emcbot
Copy link
Copy Markdown

emcbot commented Nov 20, 2024

CI Passed on Hercules in Build# 3
Built and ran in directory /work2/noaa/global/CI/HERCULES/2982


Experiment C48_ATM_3070f0fd Completed 2 Cycles: *SUCCESS* at Wed Nov 20 04:10:36 CST 2024
Experiment C96_S2SWA_gefs_replay_ics_3070f0fd Completed 1 Cycles: *SUCCESS* at Wed Nov 20 04:28:52 CST 2024
Experiment C96C48_hybatmDA_3070f0fd Completed 3 Cycles: *SUCCESS* at Wed Nov 20 05:05:30 CST 2024
Experiment C96_atm3DVar_3070f0fd Completed 3 Cycles: *SUCCESS* at Wed Nov 20 05:11:12 CST 2024
Experiment C48_S2SW_3070f0fd Completed 2 Cycles: *SUCCESS* at Wed Nov 20 05:59:34 CST 2024
Experiment C48_S2SWA_gefs_3070f0fd Completed 1 Cycles: *SUCCESS* at Wed Nov 20 06:18:30 CST 2024

Copy link
Copy Markdown
Contributor

@DavidHuber-NOAA DavidHuber-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The content looks good, just some suggestions on comments and future issues.

Comment thread ush/forecast_postdet.sh
Comment thread ush/forecast_postdet.sh
Comment thread ush/forecast_postdet.sh
Comment thread ush/forecast_predet.sh
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI-Hera-Passed **Bot use only** CI testing on Hera for this PR has completed successfully CI-Hercules-Passed **Bot use only** CI testing on Hercules for this PR has completed successfully CI-Wcoss2-Passed CI testing on WCOSS for this PR has completed successfully

Projects

None yet

Development

Successfully merging this pull request may close these issues.

GOCART AOD files not copied properly

7 participants