Update global atmos upp job to use COMIN/COMOUT by mingshichen-noaa · Pull Request #2867 · NOAA-EMC/global-workflow

mingshichen-noaa · 2024-08-27T17:11:45Z

Description

NCO has requested that each COM variable specify whether it is an input or an output. This completes that process for the global-workflow Unified Post Processor (UPP) task.

Refs #2451

Type of change

Maintenance (code refactor, clean-up, new CI test, etc.)

Change characteristics

Is this a breaking change (a change in existing functionality)? NO
Does this change require a documentation update? NO

How has this been tested?

Clone and build on RDHPCS
Cycled tests on Hercules
Forecast-only tests on Hercules

Checklist

Any dependent changes have been merged and published
My code follows the style guidelines of this project
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
My changes generate no new warnings
New and existing tests pass with my changes
I have made corresponding changes to the documentation if necessary

WalterKolczynski-NOAA

Please be sure to run one of the cycled DA tests that will exercise this job (like C96C48_hybatmDA.yaml). Look for the gdasatmanlupp and gfsatmanlupp tasks.

WalterKolczynski-NOAA · 2024-08-29T19:09:10Z

You need to undo the changes to the submodules.

When you pull in changes that update submodules, the submodules locally are not automatically updated. So when you do a git add, it commits the OLD versions back, which is definitely not what we want. Instead, you should have run git submodule update --init --recursive to update the submodules locally. Now, you need to check out the correct versions from develop and commit them (or rebase interactively to remove the offending commits, but that is a bit advanced).

aerorahul · 2024-09-03T14:30:24Z

+   COMIN_ATMOS_ANALYSIS:COM_ATMOS_ANALYSIS_TMPL \
+   COMIN_ATMOS_HISTORY:COM_ATMOS_HISTORY \
+   COMOUT_ATMOS_MASTER:COM_ATMOS_MASTER
+if [[ ! -d ${COMOUT_ATMOS_MASTER} ]]; then mkdir -m 775 -p "${COMOUT_ATMOS_MASTER}"; fi


Suggested change

if [[ ! -d ${COMOUT_ATMOS_MASTER} ]]; then mkdir -m 775 -p "${COMOUT_ATMOS_MASTER}"; fi

if [[ ! -d ${COMOUT_ATMOS_MASTER} ]]; then mkdir -p "${COMOUT_ATMOS_MASTER}"; fi

Please remove -m 775

I have removed "-m 775" on line 19.

@mingshichen-noaa
I still see -m 775 in this PR. Did you push your change?

@aerorahul
After I removed "-m 775" and then git add file, git commit -m ""-m 775" ,
finally git push origin HEAD:feature/atmos_upp_com_in_out.
Just now I tried to do the above mentioned final step again.
But after I checked jobs/JGLOBAL_ATMOS_UPP in the PR, "-m 775" is still over there. So far I have not identified the cause.

@aerorahul
I identified the reason why you see still "-m 775 in this PR". This is because I created two branches: origin/atmos_upp_com_in_out and origin/feature/atmos_upp_com_in_out. I did wrongly git push the modified files into origin/atmos_upp_com_in_out. in order to avoid the mistakes again, I have deleted the origin/atmos_upp_com_in_out.
Please review the PR again. Thanks.

aerorahul

just a note on -m 775 when creating the directory and revert the submodule pointers.
otherwise, looks good.

WalterKolczynski-NOAA · 2024-09-10T15:53:20Z

Also, even though it isn't showing as a conflict, wxflow needs to be reverted to the develop version the same way gdas and ufs_model were.

aerorahul

Looks good. Thanks.

emcbot · 2024-09-14T14:41:20Z

CI Update on Wcoss2 at 09/14/24 02:41:20 PM
============================================
Cloning and Building global-workflow PR: 2867
with PID: 106291 on host: dlogin03

emcbot · 2024-09-14T15:23:20Z

Automated global-workflow Testing Results:

Machine: Wcoss2
Start: Sat Sep 14 14:45:36 UTC 2024 on dlogin03
---------------------------------------------------
Build: Completed at 09/14/24 03:22:57 PM
Case setup: Completed for experiment C48_ATM_dddcaa81
Case setup: Skipped for experiment C48mx500_3DVarAOWCDA_dddcaa81
Case setup: Skipped for experiment C48_S2SWA_gefs_dddcaa81
Case setup: Completed for experiment C48_S2SW_dddcaa81
Case setup: Completed for experiment C96_atm3DVar_extended_dddcaa81
Case setup: Skipped for experiment C96_atm3DVar_dddcaa81
Case setup: Completed for experiment C96C48_hybatmaerosnowDA_dddcaa81
Case setup: Completed for experiment C96C48_hybatmDA_dddcaa81
Case setup: Completed for experiment C96C48_ufs_hybatmDA_dddcaa81

emcbot · 2024-09-15T04:45:10Z

All CI Test Cases Passed on Wcoss2:

Experiment C48_ATM_dddcaa81 *** SUCCESS *** at 09/14/24 04:57:13 PM
Experiment C48_S2SW_dddcaa81 *** SUCCESS *** at 09/14/24 05:06:14 PM
Experiment C96C48_hybatmDA_dddcaa81 *** SUCCESS *** at 09/14/24 06:03:27 PM
Experiment C96C48_hybatmaerosnowDA_dddcaa81 *** SUCCESS *** at 09/14/24 06:57:25 PM
Experiment C96C48_ufs_hybatmDA_dddcaa81 *** SUCCESS *** at 09/14/24 08:06:15 PM
Experiment C96_atm3DVar_extended_dddcaa81 *** SUCCESS *** at 09/15/24 04:42:34 AM

emcbot · 2024-09-16T06:12:53Z

CI Passed on Hercules in Build# 6
Built and ran in directory /work2/noaa/stmp/CI/HERCULES/2867


Experiment C48_ATM_dddcaa81 Completed 1 Cycles: *SUCCESS* at Sun Sep 15 23:10:58 CDT 2024
Experiment C96_atm3DVar_dddcaa81 Completed 3 Cycles: *SUCCESS* at Mon Sep 16 00:11:35 CDT 2024
Experiment C96C48_hybatmDA_dddcaa81 Completed 3 Cycles: *SUCCESS* at Mon Sep 16 00:17:48 CDT 2024
Experiment C48_S2SW_dddcaa81 Completed 1 Cycles: *SUCCESS* at Mon Sep 16 00:47:58 CDT 2024
Experiment C48_S2SWA_gefs_dddcaa81 Completed 1 Cycles: *SUCCESS* at Mon Sep 16 01:12:39 CDT 2024

mingshichen-noaa · 2024-09-16T14:01:48Z

@aerorahul @WalterKolczynski-NOAA
Thank you very much for your help on #2867 approved.

mingshichen-noaa and others added 8 commits August 12, 2024 09:53

changing COM_ into COMIN

91969c3

changing COM_ into COMIN_

f93185c

change COMIN_ATMOS_MASTER into COMOUT_ATMOS_MASTER

e4446c6

change COMIN_ATMOS_MASTER into COMOUT_ATMOS_MASTER

d842dd8

changing COM/ into COMIN/ or COMOUT/

f296840

update COM_ into COMIN_ or COMOUT_

ba61fad

Merge branch 'NOAA-EMC:develop' into feature/atmos_upp_com_in_out

0be05d7

Merge branch 'NOAA-EMC:develop' into feature/atmos_upp_com_in_out

435bdd7

mingshichen-noaa requested review from DavidHuber-NOAA, WalterKolczynski-NOAA and aerorahul August 27, 2024 17:13

WalterKolczynski-NOAA requested changes Aug 28, 2024

View reviewed changes

Comment thread jobs/JGLOBAL_ATMOS_UPP Outdated

Comment thread ush/python/pygfs/task/upp.py Outdated

Comment thread ush/python/pygfs/task/upp.py Outdated

mingshichen-noaa added 5 commits August 28, 2024 14:48

change COM to COMIN or COMOUT with explicitly specifying the template

668cfbd

gdas.cd and ufs_model.fd have been updated

4b08226

remove COMIN, only keep COMOUT

06ed674

modified indent space

70d6e15

update gdas.cd wxflow

09871b2

aerorahul reviewed Sep 3, 2024

View reviewed changes

mingshichen-noaa added 2 commits September 10, 2024 09:59

recover the add gdas.cd ufs_model.fd

5c74d5c

resolved file conflict of upp.yaml

0aeef60

WalterKolczynski-NOAA reviewed Sep 10, 2024

View reviewed changes

Comment thread parm/post/upp.yaml Outdated

mingshichen-noaa added 3 commits September 10, 2024 11:38

recover wxflow

99536e1

removed -m 755

f2c099f

resolve conflicts of upp.yaml

c459290

aerorahul previously approved these changes Sep 11, 2024

View reviewed changes

WalterKolczynski-NOAA approved these changes Sep 11, 2024

View reviewed changes

WalterKolczynski-NOAA added the CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera label Sep 11, 2024

aerorahul reviewed Sep 14, 2024

View reviewed changes

Comment thread parm/post/upp.yaml Outdated

Update parm/post/upp.yaml

90f67ab

aerorahul dismissed WalterKolczynski-NOAA’s stale review via 90f67ab September 14, 2024 14:39

aerorahul added CI-Wcoss2-Ready PR is ready for CI testing on WCOSS2. and removed CI-Wcoss2-Failed CI testing on WCOSS for this PR has failed labels Sep 14, 2024

Merge branch 'develop' into feature/atmos_upp_com_in_out

dddcaa8

emcbot added CI-Wcoss2-Building CI testing is cloning/building on WCOSS2 and removed CI-Wcoss2-Ready PR is ready for CI testing on WCOSS2. labels Sep 14, 2024

emcbot added CI-Wcoss2-Running CI testing on WCOSS for this PR is in-progress and removed CI-Wcoss2-Building CI testing is cloning/building on WCOSS2 labels Sep 14, 2024

emcbot added CI-Wcoss2-Passed CI testing on WCOSS for this PR has completed successfully and removed CI-Wcoss2-Running CI testing on WCOSS for this PR is in-progress labels Sep 15, 2024

WalterKolczynski-NOAA added CI-Hercules-Ready **CM use only** PR is ready for CI testing on Hercules and removed CI-Hercules-Failed **Bot use only** CI testing on Hercules for this PR has failed labels Sep 16, 2024

Merge branch 'develop' into feature/atmos_upp_com_in_out

970998c

aerorahul requested review from WalterKolczynski-NOAA and aerorahul September 16, 2024 13:56

aerorahul approved these changes Sep 16, 2024

View reviewed changes

aerorahul merged commit 2602eac into NOAA-EMC:develop Sep 16, 2024

	if [[ ! -d ${COMOUT_ATMOS_MASTER} ]]; then mkdir -m 775 -p "${COMOUT_ATMOS_MASTER}"; fi
	if [[ ! -d ${COMOUT_ATMOS_MASTER} ]]; then mkdir -p "${COMOUT_ATMOS_MASTER}"; fi

Conversation

mingshichen-noaa commented Aug 27, 2024

Description

Type of change

Change characteristics

How has this been tested?

Checklist

Uh oh!

WalterKolczynski-NOAA left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

WalterKolczynski-NOAA commented Aug 29, 2024

Uh oh!

aerorahul Sep 3, 2024

Choose a reason for hiding this comment

Uh oh!

mingshichen-noaa Sep 3, 2024

Choose a reason for hiding this comment

Uh oh!

aerorahul Sep 11, 2024

Choose a reason for hiding this comment

Uh oh!

mingshichen-noaa Sep 11, 2024

Choose a reason for hiding this comment

Uh oh!

mingshichen-noaa Sep 12, 2024

Choose a reason for hiding this comment

Uh oh!

aerorahul left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

WalterKolczynski-NOAA commented Sep 10, 2024

Uh oh!

aerorahul left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

emcbot commented Sep 14, 2024

Uh oh!

emcbot commented Sep 14, 2024

Uh oh!

emcbot commented Sep 15, 2024

Uh oh!

emcbot commented Sep 16, 2024

Uh oh!

mingshichen-noaa commented Sep 16, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants