Parse config files with input envars

[DavidHuber-NOAA]

Description

This allows the preconditioned sourcing of scripts based on optional environmental variables passed to the parse_config method within the Configuration class. A new test is added for the feature.

This is a required feature for NOAA-EMC/global-workflow#2693 to be able to source configs with RUN specified in order to get RUN-specific resources.

Type of change

• New feature (non-breaking change which adds functionality)
• This change requires a documentation update

How Has This Been Tested?

Created an xml with global-workflow branch https://github.com/DavidHuber-NOAA/global-workflow/tree/feature/simplify_res

• pynorms
• pytests

Checklist

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• My changes need updates to the documentation. I have made corresponding changes to the documentation
• My changes generate no new warnings
• New and existing tests pass with my changes
• Any dependent changes have been merged and published

[codecov]

Codecov Report

Attention: Patch coverage is 66.66667% with 3 lines in your changes missing coverage. Please review.

Project coverage is 49.93%. Comparing base (5dad7dd) to head (29c1a3c).
Report is 6 commits behind head on develop.

Files with missing lines	Patch %	Lines
src/wxflow/configuration.py	66.66%	2 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop      #35      +/-   ##
===========================================
+ Coverage    48.23%   49.93%   +1.70%     
===========================================
  Files           18       18              
  Lines         1644     1650       +6     
  Branches       335      337       +2     
===========================================
+ Hits           793      824      +31     
+ Misses         791      765      -26     
- Partials        60       61       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[aerorahul]

Is that really appropriate for a method that is supposed to just source a shell script and return the set variables as a dictionary.

This PR extends that from parsing/sourcing shell scripts and adding additional variables to that dictionary.

IMO that should be done after the sourcing and adding/updating the envvars.

[DavidHuber-NOAA]

@aerorahul I agree with your assessment on the capability. However, modifying the variables after sourcing is not enough for the global-workflow resources. They depend on knowing the RUN at the time of sourcing, otherwise those variables will be set based on the default RUN (gfs).

I see one possible solution:
Read the variables in the input config(s) and check for the existence of the input envvars. If they do not exist, raise an error. If they do exist, export the input envvars and source again.

An option that seems viable but is not:
In the global workflow at setup time, source config.base once, then modify the RUN in memory before sourcing task configs. Unfortunately, this will not work unless the contents of config.base are exported to bash as sourcing the task configs requires knowing the contents of config.base. This leads us back to a wxflow modification.

[DavidHuber-NOAA]

After offline discussion with @aerorahul, found a solution in the global-workflow to handle this case. Closing.

[DavidHuber-NOAA]

After testing of another option, this was found to be the favorable one. Reopening.

[DavidHuber-NOAA]

Pytests were run on WCOSS2 successfully.

[aerorahul]

Looks good.