Stand alone script to run a single atm 3DVar analysis

[CoryMartin-NOAA]

This PR adds a new ush script, run_single_atm_var_analysis.py.

This script does the following:

• Reads an input user configurable YAML (see below)
• Creates a config dict matching what ush/genYAML expects
• Creates the 3DVar YAML
• Uses R2D2 to stage backgrounds, obs, bias coeff files
• Links fix files from a fix/ directory I compiled
• Links the executable to the working directory
• Sets up all additionally needed directories
• Generates a SLURM batch script and submits the job

Along with this script, the associated functions have been added/modified to support each of the above steps.

A sample input YAML:

working directory: /work2/noaa/stmp/cmartin/gdas_single_test
GDASApp home: /work2/noaa/da/cmartin/GDASApp/work/GDASApp
analysis options:
  berror_yaml: /work2/noaa/da/cmartin/GDASApp/work/GDASApp/parm/atm/berror/staticb_bump.yaml
  obs_yaml_dir: /work2/noaa/da/cmartin/GDASApp/work/GDASApp/parm/atm/obs/config
  var_yaml: /work2/noaa/da/cmartin/GDASApp/work/GDASApp/parm/atm/variational/3dvar_dripcg.yaml
  obs_list: /work2/noaa/da/cmartin/GDASApp/work/GDASApp/parm/atm/obs/lists/gdas_prototype.yaml
  gdas_fix_root: /work2/noaa/da/cmartin/GDASApp/fix
  atm: true
  layout_x: 1
  layout_y: 1
  atm_window_length: PT6H
  valid_time: 2021-12-21T06:00:00Z
  dump: gdas
  case: C96
  case_enkf: C48
  dohybvar: true
  levs: 128
job options:
  machine: orion
  account: da-cpu
  queue: debug
  partition: debug
  walltime: '10:00'
  ntasks: 6
  cpus-per-task: 1
  modulepath: /work2/noaa/da/cmartin/GDASApp/work/GDASApp/modulefiles

To run:

module use /path/to/clone/modulefiles
module load GDAS/orion
export PYTHONPATH:/path/to/clone/ush:$PYTHONPATH
python run_single_atm_var_analysis.py --config sample.yaml

Sample working directory can be found at: /work2/noaa/stmp/cmartin/gdas_single_test

[CoryMartin-NOAA]

Note dohybvar=true isn't quite correct, as it is not hybrid-3DVar, but this controls making the analysis resolution 1/2 that of the background resolution.

@FanHan-NOAA you might want to try being a beta tester here to clone, build, and run this single script, as it will help with your interpolation work for the ISDA conference.

[aerorahul]

I would suggest using scheduler instead of machine where these kind of decisions need to be made.

[CoryMartin-NOAA]

Yeah I thought about that, but wanted to make it simpler for the user. Should I do the workflow approach where scheduler is the if statement but it is determined based on a dict (if scheduler[job_config['machine']] == 'slurm')?

[CoryMartin-NOAA]

@aerorahul how is this?

[FanHan-NOAA]

Note dohybvar=true isn't quite correct, as it is not hybrid-3DVar, but this controls making the analysis resolution 1/2 that of the background resolution.

@FanHan-NOAA you might want to try being a beta tester here to clone, build, and run this single script, as it will help with your interpolation work for the ISDA conference.

Great! Thanks for this addition. I'll start testing it soon.

[RussTreadon-NOAA]

@CoryMartin-NOAA : As we review this PR shall we ignore, for the the time being, pieces specific to Hera and focus solely on Orion?

[CoryMartin-NOAA]

@RussTreadon-NOAA I'm open to see what you and others think. Currently this will only work on Orion, as I need to stage fix files and a R2D2 database on Hera. Does it make sense to support both systems now, or focus on Orion and port when things have stabilized slightly?

[RussTreadon-NOAA]

@CoryMartin-NOAA, I agree with you. To keep things moving focus on Orion and port when GDASApp has stabilized.

[RussTreadon-NOAA]

Successfully installed and ran run_single_atm_var_analysis.py on Orion. The description for PR #42 provides a sample.yaml. Should we commit a templated sample.yaml to feature/single_anl?

[CoryMartin-NOAA]

Should we commit a templated sample.yaml to feature/single_anl?

I was actually thinking the next step is to add a "regression test" that would do a nccmp on the increment file and compare it to the existing increment file and determine if the analysis still is produced. That way the regression test would hold the sample YAML file, but I can alternatively create a parm/examples directory or something like that?

[emcbot]

Automated Pull Request Testing Results:
Machine: orion

Start: Thu Apr  7 13:45:20 CDT 2022 on Orion-login-1.HPC.MsState.Edu
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Thu Apr  7 14:27:31 CDT 2022
---------------------------------------------------
Tests:                                 *SUCCESS*
Tests: Completed at Thu Apr  7 14:28:29 CDT 2022
Tests: 100% tests passed, 0 tests failed out of 11

[RussTreadon-NOAA]

Should we commit a templated sample.yaml to feature/single_anl?

I was actually thinking the next step is to add a "regression test" that would do a nccmp on the increment file and compare it to the existing increment file and determine if the analysis still is produced. That way the regression test would hold the sample YAML file, but I can alternatively create a parm/examples directory or something like that?

What you outline is feasible. This will provide users with an example of what sample.yaml looks like.

[RussTreadon-NOAA]

Thank you, @CoryMartin-NOAA , for answering my questions. The changes look good to me.
I see that you are working with @aerorahul regarding scheduler in ush/ufsda/misc_utils.py. I'm more an observer than a reviewer when it comes to these details. It's a good learning experience for me. Thanks!

[CoryMartin-NOAA]

Thanks @RussTreadon-NOAA , I have pushed a new commit that dismissed your approval, if @aerorahul is happy with my new changes, I will merge this in once I have one 'approval'

[aerorahul]

Changes look good.

[RussTreadon-NOAA]

Looks good to me.