Stat Analysis Implementation for global-workflow

[kevindougherty-noaa]

Below are the appropriate changes to run a stat analysis job in the global-workflow.

Additional changes to GDASApp are also needed for jcb-gdas and jcb-algorithms for these scripts to run completely successfully.

[kevindougherty-noaa]

@CoryMartin-NOAA you will need to add Ed and Andrew to be able to review whenever you get a chance.

[CoryMartin-NOAA]

@ADCollard @EdwardSafford-NOAA feel free to take a look at this. I'll do the same and once we've done a review/test, we will open the PR with the authoritative global-workflow repo.

[CoryMartin-NOAA]

I think this is working now but I have a disclaimer/question for people.
Currently I have working branches of jcb-gdas and GDASApp. I'm not 100% thrilled with naming conventions on things like directory structures, etc. Do people think that I should just leave as is and fix later once this is all merged in, or do it now and keep these feature branches open for a bit? Next week I'll need to focus on obs processing for v17 so this will have to wait until the majority of the ob processing work is done.

[RussTreadon-NOAA]

My two cents is to take the iterative approach: merge what works now and revisit names/directories in the next round.

While developers may exercise the new functionality and get accustomed to the current names/directories, this is a new functionality. As such, we should expect refactoring.

While paths and names might change, developers can still benefit from having the output. They can provide feedback on the output (pros & cons) and maybe even develop tools to process the output (e.g., create plots).

[CoryMartin-NOAA]

Thanks @RussTreadon-NOAA ok, let me open PRs in JCB-GDAS and then GDASApp, then we can start the process of getting this initial capability working in develop of the workflow

[RussTreadon-NOAA]

WCOSS2 g-w CI
Install feature/stat-analysis at b94ce4b on Dogwood. Run g-w CI with the following results

/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C48_ATM_pr8
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202103231200        Done    Mar 17 2025 20:48:25    Mar 17 2025 23:10:16
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C48mx500_3DVarAOWCDA_pr8
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202103241800        Done    Mar 17 2025 20:48:27    Mar 17 2025 21:05:17
202103250000        Done    Mar 17 2025 20:48:27    Mar 17 2025 23:10:20
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C48mx500_hybAOWCDA_pr8
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202103241800        Done    Mar 17 2025 20:48:29    Mar 17 2025 21:05:34
202103250000        Done    Mar 17 2025 20:48:29    Mar 17 2025 23:10:23
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C48_S2SWA_gefs_pr8
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202103231200        Done    Mar 17 2025 20:48:31    Mar 17 2025 22:50:26
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C48_S2SW_pr8
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202103231200        Done    Mar 17 2025 20:48:34    Mar 17 2025 23:10:28
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C96_atm3DVar_pr8
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202112201800        Done    Mar 17 2025 20:48:36    Mar 17 2025 21:06:15
202112210000        Done    Mar 17 2025 20:48:36    Mar 17 2025 23:35:23
202112210600        Done    Mar 17 2025 20:48:36    Mar 17 2025 23:10:32
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C96C48_hybatmaerosnowDA_pr8
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202112201200        Done    Mar 17 2025 20:48:39    Mar 17 2025 21:10:41
202112201800        Done    Mar 17 2025 20:48:39    Mar 17 2025 23:30:31
202112210000        Done    Mar 17 2025 20:48:39    Mar 17 2025 23:15:33
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C96C48_hybatmDA_pr8
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202112201800        Done    Mar 17 2025 20:48:41    Mar 17 2025 21:06:26
202112210000        Done    Mar 17 2025 20:48:41    Mar 17 2025 23:10:39
202112210600        Done    Mar 17 2025 20:48:41    Mar 17 2025 23:10:39
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C96C48_ufs_hybatmDA_pr8
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202402231800        Done    Mar 17 2025 20:48:43    Mar 17 2025 21:06:33
202402240000        Done    Mar 17 2025 20:48:43    Mar 17 2025 23:45:30
202402240600        Done    Mar 17 2025 20:48:43    Mar 17 2025 23:35:30
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C96mx100_S2S_pr8
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
199405010000        Done    Mar 17 2025 20:48:45    Mar 17 2025 22:16:03

All jobs in all cases ran to completion.

A check of JEDI-based DA cases finds stat the stat jobs ran

russ.treadon@dlogin07:/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C48mx500_3DVarAOWCDA_pr8> rocotostat -d C48mx500_3DVarAOWCDA_pr8.db -w C48mx500_3DVarAOWCDA_pr8.xml |grep stat
202103250000             gfs_anlstat                    18656178           SUCCEEDED                   0         1          47.0
202103250000            gdas_anlstat                    18656179           SUCCEEDED                   0         1          47.0


russ.treadon@dlogin07:/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C48mx500_hybAOWCDA_pr8> rocotostat -d C48mx500_hybAOWCDA_pr8.db -w C48mx500_hybAOWCDA_pr8.xml |grep stat
202103250000            gdas_anlstat                    18657933           SUCCEEDED                   0         1          47.0


russ.treadon@dlogin07:/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C96C48_hybatmaerosnowDA_pr8> rocotostat -d C96C48_hybatmaerosnowDA_pr8.db -w C96C48_hybatmaerosnowDA_pr8.xml |grep stat
202112201800             gfs_anlstat                    18659314           SUCCEEDED                   0         1          62.0
202112201800            gdas_anlstat                    18659321           SUCCEEDED                   0         1          62.0
202112210000            gdas_anlstat                    18671165           SUCCEEDED                   0         1          63.0


russ.treadon@dlogin07:/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C96C48_ufs_hybatmDA_pr8> rocotostat -d C96C48_ufs_hybatmDA_pr8.db -w C96C48_ufs_hybatmDA_pr8.xml |grep stat
202402240000             gfs_anlstat                    18663812           SUCCEEDED                   0         1          61.0
202402240000            gdas_anlstat                    18663819           SUCCEEDED                   0         1          60.0
202402240600            gdas_anlstat                    18680859           SUCCEEDED                   0         1          63.0

[RussTreadon-NOAA]

Approve based on successful WCOSS2 g-w CI.

Two caveats

1. I did not check the stat jobs to ensure that the expected output is correctly generated
2. WCOSS2 g-w CI does not run archive (hpss) jobs. HPSS is only exercised when running g-w CI on Hera. If the stat jobs create file(s) written to HPSS, the WCOSS2 tests do not exercise this function.

[CoryMartin-NOAA]

Thank you @RussTreadon-NOAA , do you , @kevindougherty-noaa and @DavidNew-NOAA think this is ready to point this PR at the NOAA-EMC:global-workflow develop branch?

[DavidNew-NOAA]

Can we add "Resolves NOAA-EMC#3147" to the description?

[CoryMartin-NOAA]

@DavidNew-NOAA that will be done when this PR is opened in the authoritative repo and not in my fork

[DavidNew-NOAA]

@CoryMartin-NOAA Ah, didn't realize this was a fork

[DavidNew-NOAA]

@CoryMartin-NOAA I haven't tested this functionality, but at this point it's far polished up enough to be a PR for GW develop