GitLab CI Framework for schedule PR cases and ctests on multi hosts

[TerrenceMcGuinness-NOAA]

Description

This PR implements a GitLab CI pipeline system to support two testing modes:

1. Standard full test cases found in $HOMEgfs/dev/ci/cases/pr on a scheduled basis
2. CTest-based functional tests triggered via GitHub actions found in $HOMEgfs/dev/ctests

While simultaneously supporting Hera and Gaea C6 in an extensible framework.

More detailed information can be found in GitLab README.md file. The primary design principle is to easily add addition modes and cases on a per host basis in a singular and systematic way.

Resolves #3597
Resolves #3499

Some refactoring additions included in this PR inspired by new vertical structure :

• Replaced all evaluations of HOMEgfs_ to a CI support util (find_homegfs.py)
• Consolidated ci_utils_wrapper.sh directly into ci_utils.sh to reduce redundancy.
• Removed unused scripts supporting old BASH CI system
• Use local variable HOMEgfs_ everywhere instead of HOMEgfs which is defined to be used directly within the runtime infrastructure intrinsically.

Type of change

• Bug fix (fixes something broken)
• New feature (adds functionality)
• Maintenance (code refactor, clean-up, new CI test, etc.)

Change characteristics

• Is this a breaking change (a change in existing functionality)? NO
• Does this change require a documentation update? YES
• Does this change require an update to any of the following submodules? NO

How has this been tested?

Runthrough Scheduled launch of pipeline on Hera and Gaea C6 simultaneously:

Runthrough a launch on Hera and Gaea C6 by setting $GITHUB_API_TRIGGER == "true"

[DavidHuber-NOAA]

Changes look good. Thanks Terry!

[TerrenceMcGuinness-NOAA]

@aerorahul I found a bug in the yaml parser for BASH that inspired me to add a few Unit Tests for that and the util for getting HOMEgfs, so maybe you can take one more look at that before we merge.

rootdir: /home/runner/work/global-workflow/global-workflow/global-workflow/dev/ci/scripts/unittests
plugins: cov-6.1.1
collecting ... collected 18 items
test_create_experiment.py::test_create_experiment PASSED                 [  5%]
test_find_homegfs.py::TestFindHOMEgfs::test_find_homegfs_current_dir PASSED [ 11%]
test_find_homegfs.py::TestFindHOMEgfs::test_find_homegfs_nested_dir PASSED [ 16%]
test_find_homegfs.py::TestFindHOMEgfs::test_find_homegfs_none_start_path PASSED [ 22%]
test_find_homegfs.py::TestFindHOMEgfs::test_find_homegfs_not_found PASSED [ 27%]
test_find_homegfs.py::TestFindHOMEgfs::test_with_path_object PASSED      [ 33%]
test_find_homegfs.py::TestFindHOMEgfs::test_with_string_path PASSED      [ 38%]
test_parse_yaml.py::TestParseYAML::test_cli_basic PASSED                 [ 44%]
test_parse_yaml.py::TestParseYAML::test_cli_default_value PASSED         [ 50%]
test_parse_yaml.py::TestParseYAML::test_cli_fail_on_missing PASSED       [ 55%]
test_parse_yaml.py::TestParseYAML::test_cli_string_option PASSED         [ 61%]
test_rocotostat.py::test_rocoto_statcount PASSED                         [ 66%]
test_rocotostat.py::test_rocoto_summary PASSED                           [ 72%]
test_rocotostat.py::test_rocoto_done PASSED                              [ 77%]
test_rocotostat.py::test_rocoto_stalled PASSED                           [ 83%]
test_setup.py::test_setup_expt PASSED                                    [ 88%]
test_setup.py::test_setup_xml PASSED                                     [ 94%]
test_setup.py::test_setup_xml_fail_config_env_cornercase PASSED          [100%]
- generated xml file: /home/runner/work/global-workflow/global-workflow/global-workflow/dev/ci/scripts/unittests/test-results.xml -
============================= 18 passed in 20.[58](https://github.com/NOAA-EMC/global-workflow/actions/runs/14707763201/job/41272078601?pr=3603#step:8:59)s =============================

[aerorahul]

lgtm