[develop] Add upgrade option to check_for_preexist_dir.

[danielabdi-noaa]

DESCRIPTION OF CHANGES:

Addresses issue #655. Add upgrade option besides delete, rename, quit when checking for existing experiment directory (or another folder). For EXPTDIR the behavior is different from other directories. The "upgrade" option behaves like rsync for EXPTDIR. The workflow generation is handled by python code in SRW app, therefore check_for_preexist_dir.py handles that. For other directories, the shell version of the script is used, and upgrade means reuse existing directory. Apparently this option is sometimes useful for RRFS_dev1 workflow runs.

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

TESTS CONDUCTED:

• hera.intel
• orion.intel
• cheyenne.intel
• cheyenne.gnu
• gaea.intel
• jet.intel
• wcoss2.intel
• NOAA Cloud (indicate which platform)
• Jenkins
• fundamental test suite
• comprehensive tests (specify which if a subset was used)

DEPENDENCIES:

DOCUMENTATION:

ISSUE:

• Upgrade EXPTDIR directory option. #655

CHECKLIST

• My code follows the style guidelines in the Contributor's Guide
• I have performed a self-review of my own code using the Code Reviewer's Guide
• I have commented my code, particularly in hard-to-understand areas
• My changes need updates to the documentation. I have made corresponding changes to the documentation
• My changes do not require updates to the documentation (explain).
• My changes generate no new warnings
• New and existing tests pass with my changes
• Any dependent changes have been merged and published

LABELS (optional):

A Code Manager needs to add the following labels to this PR:

• Work In Progress
• bug
• enhancement
• documentation
• release
• high priority
• run_ci
• run_we2e_fundamental_tests
• run_we2e_comprehensive_tests
• Needs Cheyenne test
• Needs Jet test
• Needs Hera test
• Needs Orion test
• help wanted

CONTRIBUTORS (optional):

[MichaelLueken]

@danielabdi-noaa These changes look good to me! I was able to clone this work on Jet, submit the fundamental WE2E tests, then reran them using the new reuse method. The new method worked as described in the PR. All tests for both runs successfully passed. Approving these changes now.

[MichaelLueken]

@danielabdi-noaa The Jenkins tests on Cheyenne Intel have finally passed successfully. Moving forward with merging this work now.