Update to obsproc/v1.2.0 and prepobs/v1.1.0 by KateFriedman-NOAA · Pull Request #2903 · NOAA-EMC/global-workflow

KateFriedman-NOAA · 2024-09-11T18:26:00Z

Description

This PR updates the develop branch to use the newer operational obsproc/v1.2.0 and prepobs/v1.1.0. The obsproc/prepobs installs in glopara space on supported platforms use tags cut from the dev/gfsv17 branches in the respective repos. The installation of prepobs/v1.1.0 on WCOSS2 is called "gfsv17_v1.1.0" to help avoid GFSv16 users using it instead of the operational module.

Also, the HOMEobsproc path is updated to set an empty default for obsproc_run_ver. This both removes the need to set a default (and constantly update it, which is duplication) and avoid the unset variable error when the fcst jobs use their own load module script that does not know obsproc_run_ver:

export HOMEobsproc="${BASE_GIT:-}/obsproc/v${obsproc_run_ver:-}"

This PR also reverts the prepobs and fit2obs installs on MSU back to the glopara space from the temporary /work/noaa/global/kfriedma/glopara space installs.

Lastly, this PR also includes updates to complete issue #2844 (merge build.spack.ver and run.spack.ver).

Resolves #2291
Resolves #2840
Resolves #2844

Type of change

Regular updates to upstream components

Change characteristics

Is this a breaking change (a change in existing functionality)? NO
Does this change require a documentation update? NO
Does this change require an update to any of the following submodules? NO (If YES, please add a link to any PRs that are pending.)

How has this been tested?

Cycled tests on WCOSS2 and Hera.

- Update to obsproc/v1.2.0 and prepobs/v1.1.0 - Revert back to glopara installs on Orion/Hercules - Remove default version for obsproc in config.base Refs NOAA-EMC#2291 Refs NOAA-EMC#2840 Refs NOAA-EMC#2841

Change Orion BASE_GIT to the same as Hercules (git_rocky9). Refs NOAA-EMC#2291 Refs NOAA-EMC#2840

Add ":-" to obsproc_run_ver variable setting for HOMEobsproc; the run.spack.ver version file is not currently sourced during forecast jobs and thus obsproc_run_ver is not set; it's not needed for forecast jobs however so allow it to be unset Refs NOAA-EMC#2291

Update module_base.gaea.lua to use the prepobs_run_ver instead of a hardcoded version. Refs NOAA-EMC#2840

Install of prepobs/v1.1.0 on WCOSS2 was installed under the folder name "1.1.0_gfsv17" to keep the v16 package from using it and to differentiate it from the ops version. Refs NOAA-EMC#2840

Refs NOAA-EMC#2840

This helps GFSv16 users avoid using this install instead of the operational module version. Refs NOAA-EMC#2840

aerorahul

looks ok

DavidHuber-NOAA

Looks good. I see that Jet is still using /lfs4 for Fit2Obs, but I think this is being covered in #2878, correct?

KateFriedman-NOAA · 2024-09-11T18:35:34Z

Awaiting install of prepobs/v1.1.0 on MSU before CI tests should start.

aerorahul · 2024-09-11T18:36:27Z

Before tests are run, do you see an issue in perhaps combining build.spack.ver and run.spack.ver into a single spack.ver and updating the host specific build and run version files?
A little busy work, but small enough and relevant to the changes in this PR.
Thoughts?

KateFriedman-NOAA · 2024-09-11T18:36:40Z

Looks good. I see that Jet is still using /lfs4 for Fit2Obs, but I think this is being covered in #2878, correct?

Yes, I believe @InnocentSouopgui-NOAA will be updating that path for Fit2Obs on Jet. They are likely also updating the prepobs path too but this PR will do that before their PR.

KateFriedman-NOAA · 2024-09-11T18:38:11Z

Before tests are run, do you see an issue in perhaps combining build.spack.ver and run.spack.ver into a single spack.ver and updating the host specific build and run version files? A little busy work, but small enough and relevant to the changes in this PR. Thoughts?

Sure, I can do that in this PR. Stay tuned...

KateFriedman-NOAA · 2024-09-11T18:39:13Z

Will now combine changes for issue #2844 into this PR.

Update machine ver files to source combined spack.ver Refs NOAA-EMC#2844

DavidHuber-NOAA

*.spack.ver merge looks good!

aerorahul

lgtm

emcbot · 2024-09-17T18:01:47Z

CI Passed on Hercules in Build# 4
Built and ran in directory /work2/noaa/stmp/CI/HERCULES/2903


Experiment C48_ATM_9a8a9001 Completed 1 Cycles: *SUCCESS* at Tue Sep 17 11:35:32 CDT 2024
Experiment C48_S2SW_9a8a9001 Completed 1 Cycles: *SUCCESS* at Tue Sep 17 12:54:51 CDT 2024
Experiment C96_atm3DVar_9a8a9001 Completed 3 Cycles: *SUCCESS* at Tue Sep 17 12:55:30 CDT 2024
Experiment C48_S2SWA_gefs_9a8a9001 Completed 1 Cycles: *SUCCESS* at Tue Sep 17 13:01:01 CDT 2024
Experiment C96C48_hybatmDA_9a8a9001 Completed 3 Cycles: *SUCCESS* at Tue Sep 17 13:01:31 CDT 2024

emcbot · 2024-09-17T18:41:55Z

Checkout Failed on Hera in Build# 6: Could not perform submodule update

emcbot · 2024-09-17T18:57:19Z

Checkout Failed on Hera in Build# 7: Could not perform submodule update

emcbot · 2024-09-18T00:11:46Z

CI Passed on Hera in Build# 8
Built and ran in directory /scratch1/NCEPDEV/global/CI/2903


Experiment C48mx500_3DVarAOWCDA_9a8a9001 Completed 2 Cycles: *SUCCESS* at Tue Sep 17 21:50:09 UTC 2024
Experiment C48_ATM_9a8a9001 Completed 1 Cycles: *SUCCESS* at Tue Sep 17 21:56:21 UTC 2024
Experiment C96C48_hybatmDA_9a8a9001 Completed 3 Cycles: *SUCCESS* at Tue Sep 17 23:03:55 UTC 2024
Experiment C96_atm3DVar_9a8a9001 Completed 3 Cycles: *SUCCESS* at Tue Sep 17 23:11:48 UTC 2024
Experiment C96C48_ufs_hybatmDA_9a8a9001 Completed 2 Cycles: *SUCCESS* at Tue Sep 17 23:34:25 UTC 2024
Experiment C48_S2SW_9a8a9001 Completed 1 Cycles: *SUCCESS* at Tue Sep 17 23:40:30 UTC 2024
Experiment C48_S2SWA_gefs_9a8a9001 Completed 1 Cycles: *SUCCESS* at Tue Sep 17 23:40:33 UTC 2024
Experiment C96C48_hybatmaerosnowDA_9a8a9001 Completed 3 Cycles: *SUCCESS* at Wed Sep 18 00:11:37 UTC 2024

KateFriedman-NOAA added 9 commits August 26, 2024 15:41

Update obsproc and prepobs versions

7008246

- Update to obsproc/v1.2.0 and prepobs/v1.1.0 - Revert back to glopara installs on Orion/Hercules - Remove default version for obsproc in config.base Refs NOAA-EMC#2291 Refs NOAA-EMC#2840 Refs NOAA-EMC#2841

Update Orion BASE_GIT

f36c329

Change Orion BASE_GIT to the same as Hercules (git_rocky9). Refs NOAA-EMC#2291 Refs NOAA-EMC#2840

Merge branch 'NOAA-EMC:develop' into feature/obsproc_prepobs_versions

ac1724d

Update Gaea modulefile to use prepobs variable

668254e

Update module_base.gaea.lua to use the prepobs_run_ver instead of a hardcoded version. Refs NOAA-EMC#2840

Update prepobs_run_ver for WCOSS2

f9f5a48

Install of prepobs/v1.1.0 on WCOSS2 was installed under the folder name "1.1.0_gfsv17" to keep the v16 package from using it and to differentiate it from the ops version. Refs NOAA-EMC#2840

Merge branch 'NOAA-EMC:develop' into feature/obsproc_prepobs_versions

14e43e1

Remove "_gfsv17" from the prepobs version

9d0edb1

Refs NOAA-EMC#2840

Add "gfsv17_" to prepobs path before version

77f16e2

This helps GFSv16 users avoid using this install instead of the operational module version. Refs NOAA-EMC#2840

KateFriedman-NOAA requested review from DavidHuber-NOAA, WalterKolczynski-NOAA and aerorahul September 11, 2024 18:26

aerorahul previously approved these changes Sep 11, 2024

View reviewed changes

KateFriedman-NOAA marked this pull request as ready for review September 11, 2024 18:34

DavidHuber-NOAA previously approved these changes Sep 11, 2024

View reviewed changes

Merge build.spack.ver and run.spack.ver

c225660

Update machine ver files to source combined spack.ver Refs NOAA-EMC#2844

KateFriedman-NOAA dismissed stale reviews from DavidHuber-NOAA and aerorahul via c225660 September 11, 2024 19:01

DavidHuber-NOAA previously approved these changes Sep 11, 2024

View reviewed changes

KateFriedman-NOAA requested a review from aerorahul September 11, 2024 19:19

aerorahul previously approved these changes Sep 11, 2024

View reviewed changes

KateFriedman-NOAA added CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera CI-Wcoss2-Ready PR is ready for CI testing on WCOSS2. labels Sep 12, 2024

Merge branch 'NOAA-EMC:develop' into feature/obsproc_prepobs_versions

258f5d3

emcbot removed the CI-Wcoss2-Ready PR is ready for CI testing on WCOSS2. label Sep 12, 2024

emcbot added CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera and removed CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera labels Sep 17, 2024

KateFriedman-NOAA mentioned this pull request Sep 17, 2024

Move to contrib installation of spack-stack on Jet #2878

Merged

23 tasks

emcbot added CI-Hera-Failed **Bot use only** CI testing on Hera for this PR has failed and removed CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera labels Sep 17, 2024

KateFriedman-NOAA added CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera and removed CI-Hera-Failed **Bot use only** CI testing on Hera for this PR has failed labels Sep 17, 2024

emcbot added CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera and removed CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera labels Sep 17, 2024

emcbot added CI-Hera-Failed **Bot use only** CI testing on Hera for this PR has failed and removed CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera labels Sep 17, 2024

TerrenceMcGuinness-NOAA added CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera and removed CI-Hera-Failed **Bot use only** CI testing on Hera for this PR has failed labels Sep 17, 2024

WalterKolczynski-NOAA approved these changes Sep 18, 2024

View reviewed changes

WalterKolczynski-NOAA merged commit 7588d2b into NOAA-EMC:develop Sep 18, 2024

KateFriedman-NOAA deleted the feature/obsproc_prepobs_versions branch September 18, 2024 12:18

Conversation

KateFriedman-NOAA commented Sep 11, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Type of change

Change characteristics

How has this been tested?

Uh oh!

aerorahul left a comment

Choose a reason for hiding this comment

Uh oh!

DavidHuber-NOAA left a comment

Choose a reason for hiding this comment

Uh oh!

KateFriedman-NOAA commented Sep 11, 2024

Uh oh!

aerorahul commented Sep 11, 2024

Uh oh!

KateFriedman-NOAA commented Sep 11, 2024

Uh oh!

KateFriedman-NOAA commented Sep 11, 2024

Uh oh!

KateFriedman-NOAA commented Sep 11, 2024

Uh oh!

DavidHuber-NOAA left a comment

Choose a reason for hiding this comment

Uh oh!

aerorahul left a comment

Choose a reason for hiding this comment

Uh oh!

emcbot commented Sep 17, 2024

Uh oh!

emcbot commented Sep 17, 2024

Uh oh!

emcbot commented Sep 17, 2024

Uh oh!

emcbot commented Sep 18, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

KateFriedman-NOAA commented Sep 11, 2024 •

edited

Loading