Update gdas.cd by guillaumevernieres · Pull Request #2978 · NOAA-EMC/global-workflow

guillaumevernieres · 2024-10-03T20:18:48Z

Description

Updates to the gdas.cd #.
@RussTreadon-NOAA will submit a PR in the GDASApp, we'll update the gdas.cd # in this branch after the GDASApp PR is merged.
In the mean time, could somebody review the few simple code changes that are needed to run with the new #?

Depends on GDASApp PR #1310
Depends on g-w issue #3012

Type of change

Maintenance (code refactor, clean-up, new CI test, etc.)

Change characteristics

Is this a breaking change (a change in existing functionality)? NO
Does this change require a documentation update? NO
Does this change require an update to any of the following submodules? YES
- GDAS NOAA-EMC/GDASApp/issues/1298 - merged

How has this been tested?

Run subset of g-w CI on Hera, Hercules, Orion, and WCOSS2 (Cactus)

Checklist

My code follows the style guidelines of this project
I have performed a self-review of my own code
This change is covered by an existing CI test or a new one has been added

aerorahul

looks good to me

RussTreadon-NOAA · 2024-10-04T18:14:24Z

@guillaumevernieres , the hash for sorc/gdas.cd can be updated. GDASApp PR #1310 was merged into develop at 9d95c9d.

Co-authored-by: Rahul Mahajan <aerorahul@users.noreply.github.com>

guillaumevernieres · 2024-10-07T13:14:35Z

@guillaumevernieres , the hash for sorc/gdas.cd can be updated. GDASApp PR #1310 was merged into develop at 9d95c9d.

Thanks @RussTreadon-NOAA , working on it this morning.

RussTreadon-NOAA · 2024-10-25T23:05:00Z

Hercules /work2/noaa/stmp usage remains over quota.

        Project: stmp
                Directory: /work/noaa/stmp DiskInUse=321861 GB, Quota=342000 GB, Files=28126233, FileQUota=0
                Directory: /work2/noaa/stmp DiskInUse=353832 GB, Quota=342000 GB, Files=47305420, FileQUota=0

Can we change GFS_CI_ROOT in ci/platforms/config.hercules to point at /work/noaa/stmp? Otherwise, we may be unable to run CI-Hercules until early next week.

WalterKolczynski-NOAA · 2024-10-25T23:14:11Z

Hercules /work2/noaa/stmp usage remains over quota.
        Project: stmp
                Directory: /work/noaa/stmp DiskInUse=321861 GB, Quota=342000 GB, Files=28126233, FileQUota=0
                Directory: /work2/noaa/stmp DiskInUse=353832 GB, Quota=342000 GB, Files=47305420, FileQUota=0
Can we change GFS_CI_ROOT in ci/platforms/config.hercules to point at /work/noaa/stmp? Otherwise, we may be unable to run CI-Hercules until early next week.

Only @TerrenceMcGuinness-NOAA can change it, and since it is now the weekend, it probably has to wait until Monday 🙁.

RussTreadon-NOAA · 2024-10-25T23:18:15Z

Hercules /work2/noaa/stmp usage remains over quota.
        Project: stmp
                Directory: /work/noaa/stmp DiskInUse=321861 GB, Quota=342000 GB, Files=28126233, FileQUota=0
                Directory: /work2/noaa/stmp DiskInUse=353832 GB, Quota=342000 GB, Files=47305420, FileQUota=0
Can we change GFS_CI_ROOT in ci/platforms/config.hercules to point at /work/noaa/stmp? Otherwise, we may be unable to run CI-Hercules until early next week.
Only @TerrenceMcGuinness-NOAA can change it, and since it is now the weekend, it probably has to wait until Monday 🙁.

I see that ci/platforms/config.orion also uses /work2/noaa/stmp. 😞

TerrenceMcGuinness-NOAA · 2024-10-25T23:46:29Z

@RussTreadon-NOAA I cleared out all the directories that I'm using under /work2/noaa/stmp for Global Workflow CI. If you have any full paths that I missed let me know.

Oh wait I hear what you're saying. We should change where it runs where we have more space, right?
Yeah we can do that. @WalterKolczynski-NOAA where is a good spot where we can redirect the output on Hercules?

WalterKolczynski-NOAA · 2024-10-26T00:10:03Z

@RussTreadon-NOAA Terry says if you change custom_workspace entry in the ci/Jenkinsfile file, it will use the new location. If you make that change, I will reset the label to try again.

Please change it to /work2/noaa/global/CI/HERCULES.

RussTreadon-NOAA · 2024-10-26T00:25:29Z

@RussTreadon-NOAA Terry says if you change custom_workspace entry in the ci/Jenkinsfile file, it will use the new location. If you make that change, I will reset the label to try again.

Please change it to /work2/noaa/global/CI/HERCULES.

You mean /work/noaa/global/CI/HERCULES, right? The file already has /work2/noaa/global/CI/HERCULES.

WalterKolczynski-NOAA · 2024-10-26T01:56:03Z

@RussTreadon-NOAA Terry says if you change custom_workspace entry in the ci/Jenkinsfile file, it will use the new location. If you make that change, I will reset the label to try again.
Please change it to /work2/noaa/global/CI/HERCULES.

You mean /work/noaa/global/CI/HERCULES, right? The file already has /work2/noaa/global/CI/HERCULES.

The file has /work2/noaa/stmp/CI/HERCULES. We just want to change the stmp to global.

RussTreadon-NOAA · 2024-10-26T10:25:37Z

@WalterKolczynski-NOAA : The change @TerrenceMcGuinness-NOAA suggested has been made. custom_workspace in ci/Jenkinsfile now has /work/noaa/stmp/CI/HERCULES. Done at 1e3398a.

WalterKolczynski-NOAA · 2024-10-26T18:48:20Z

All tests passed on WCOSS.

emcbot · 2024-10-26T19:35:32Z

Experiment C48_ATM FAILED on Hercules in Build# 5 in
/work/noaa/stmp/CI/HERCULES/2978/RUNTESTS/EXPDIR/

emcbot · 2024-10-26T19:35:32Z

Experiment C96C48_hybatmDA FAILED on Hercules in Build# 5 in
/work/noaa/stmp/CI/HERCULES/2978/RUNTESTS/EXPDIR/

emcbot · 2024-10-26T19:35:34Z

Experiment C96_S2SWA_gefs_replay_ics FAILED on Hercules in Build# 5 in
/work/noaa/stmp/CI/HERCULES/2978/RUNTESTS/EXPDIR/

emcbot · 2024-10-26T19:35:34Z

Experiment C48_S2SWA_gefs FAILED on Hercules in Build# 5 in
/work/noaa/stmp/CI/HERCULES/2978/RUNTESTS/EXPDIR/

emcbot · 2024-10-26T19:35:35Z

Experiment C96_atm3DVar FAILED on Hercules in Build# 5 in
/work/noaa/stmp/CI/HERCULES/2978/RUNTESTS/EXPDIR/

emcbot · 2024-10-26T19:35:36Z

Experiment C48_S2SW FAILED on Hercules in Build# 5 in
/work/noaa/stmp/CI/HERCULES/2978/RUNTESTS/EXPDIR/

WalterKolczynski-NOAA · 2024-10-26T19:49:16Z

stmp on /work is also full. Needs to be /work2/noaa/global/CI/HERCULES like I requested.

RussTreadon-NOAA · 2024-10-26T21:04:02Z

@WalterKolczynski-NOAA . I failed to correctly read your message. 17cf1eb commits the change you requested. custom_workspace is now set to /work2/noaa/global for Hercules in ci/Jenkinsfile

RussTreadon-NOAA · 2024-10-26T21:05:24Z

@WalterKolczynski-NOAA , feel free to edit ci/Jenkinsfile in guillaumevernieres:feature/update_hashes as you wish. This isn't a file users should normally edit, right?

emcbot · 2024-10-26T21:49:58Z

Experiment C96C48_hybatmDA FAILED on Hercules in Build# 6 in
/work2/noaa/global/CI/HERCULES/2978/RUNTESTS/EXPDIR/

emcbot · 2024-10-26T21:49:58Z

Experiment C48_ATM FAILED on Hercules in Build# 6 in
/work2/noaa/global/CI/HERCULES/2978/RUNTESTS/EXPDIR/

emcbot · 2024-10-26T21:49:59Z

Experiment C48_S2SWA_gefs FAILED on Hercules in Build# 6 in
/work2/noaa/global/CI/HERCULES/2978/RUNTESTS/EXPDIR/

emcbot · 2024-10-26T21:50:00Z

Experiment C96_S2SWA_gefs_replay_ics FAILED on Hercules in Build# 6 in
/work2/noaa/global/CI/HERCULES/2978/RUNTESTS/EXPDIR/

emcbot · 2024-10-26T21:50:00Z

Experiment C48_S2SW FAILED on Hercules in Build# 6 in
/work2/noaa/global/CI/HERCULES/2978/RUNTESTS/EXPDIR/

emcbot · 2024-10-26T21:50:02Z

Experiment C96_atm3DVar FAILED on Hercules in Build# 6 in
/work2/noaa/global/CI/HERCULES/2978/RUNTESTS/EXPDIR/

WalterKolczynski-NOAA · 2024-10-26T21:59:21Z

Okay, there are other issues with CI going on beyond just full drives. Going to go ahead and merge this. Will keep the new CI location, so there are fewer people to harangue to clean up space when we run out.

RussTreadon-NOAA · 2024-10-27T00:07:24Z

Thanks you @WalterKolczynski-NOAA for merging this PR into develop.

guillaumevernieres added 3 commits October 2, 2024 18:30

wip, added tools to select mom6 levels

84600b6

pynorm ...

303d417

switched to dict

24839f5

guillaumevernieres requested review from CoryMartin-NOAA and RussTreadon-NOAA October 3, 2024 20:19

CoryMartin-NOAA previously approved these changes Oct 3, 2024

View reviewed changes

RussTreadon-NOAA mentioned this pull request Oct 4, 2024

Update select JEDI hashes NOAA-EMC/GDASApp#1310

Merged

aerorahul previously approved these changes Oct 4, 2024

View reviewed changes

aerorahul reviewed Oct 4, 2024

View reviewed changes

Comment thread ush/python/pygfs/utils/marine_da_utils.py

guillaumevernieres changed the title ~~Feature/update hashes~~ Update gdas.cd Oct 7, 2024

Update ush/python/pygfs/utils/marine_da_utils.py

9538ba5

Co-authored-by: Rahul Mahajan <aerorahul@users.noreply.github.com>

guillaumevernieres dismissed stale reviews from aerorahul and CoryMartin-NOAA via 9538ba5 October 7, 2024 13:13

new jedi hashes

4a3d981

guillaumevernieres marked this pull request as ready for review October 7, 2024 13:21

guillaumevernieres requested review from CoryMartin-NOAA and aerorahul October 7, 2024 13:23

CoryMartin-NOAA previously approved these changes Oct 7, 2024

View reviewed changes

aerorahul previously approved these changes Oct 7, 2024

View reviewed changes

aerorahul added CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera CI-Orion-Ready **CM use only** PR is ready for CI testing on Orion CI-Wcoss2-Ready PR is ready for CI testing on WCOSS2. labels Oct 7, 2024

change hercules custom_workspace to /work/noaa/stmp in Jenkinsfile

1e3398a

change hercules custom_workspace to /work2/noaa/global in Jenkinsfile

17cf1eb

WalterKolczynski-NOAA approved these changes Oct 26, 2024

View reviewed changes

RussTreadon-NOAA mentioned this pull request Oct 29, 2024

Resume nightly testing of GDASApp NOAA-EMC/GDASApp#1313

Closed

Conversation

guillaumevernieres commented Oct 3, 2024 • edited by RussTreadon-NOAA Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Type of change

Change characteristics

How has this been tested?

Checklist

Uh oh!

aerorahul left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

RussTreadon-NOAA commented Oct 4, 2024

Uh oh!

guillaumevernieres commented Oct 7, 2024

Uh oh!

RussTreadon-NOAA commented Oct 25, 2024

Uh oh!

WalterKolczynski-NOAA commented Oct 25, 2024

Uh oh!

RussTreadon-NOAA commented Oct 25, 2024

Uh oh!

TerrenceMcGuinness-NOAA commented Oct 25, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

WalterKolczynski-NOAA commented Oct 26, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

RussTreadon-NOAA commented Oct 26, 2024

Uh oh!

WalterKolczynski-NOAA commented Oct 26, 2024

Uh oh!

RussTreadon-NOAA commented Oct 26, 2024

Uh oh!

WalterKolczynski-NOAA commented Oct 26, 2024

Uh oh!

emcbot commented Oct 26, 2024

Uh oh!

emcbot commented Oct 26, 2024

Uh oh!

emcbot commented Oct 26, 2024

Uh oh!

emcbot commented Oct 26, 2024

Uh oh!

emcbot commented Oct 26, 2024

Uh oh!

emcbot commented Oct 26, 2024

Uh oh!

WalterKolczynski-NOAA commented Oct 26, 2024

Uh oh!

RussTreadon-NOAA commented Oct 26, 2024

Uh oh!

RussTreadon-NOAA commented Oct 26, 2024

Uh oh!

emcbot commented Oct 26, 2024

Uh oh!

emcbot commented Oct 26, 2024

Uh oh!

emcbot commented Oct 26, 2024

Uh oh!

emcbot commented Oct 26, 2024

Uh oh!

emcbot commented Oct 26, 2024

Uh oh!

emcbot commented Oct 26, 2024

Uh oh!

WalterKolczynski-NOAA commented Oct 26, 2024

Uh oh!

RussTreadon-NOAA commented Oct 27, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

11 participants

guillaumevernieres commented Oct 3, 2024 •

edited by RussTreadon-NOAA

Loading

TerrenceMcGuinness-NOAA commented Oct 25, 2024 •

edited

Loading

WalterKolczynski-NOAA commented Oct 26, 2024 •

edited

Loading