Add diffusion/diag B for aerosol DA and some other needed changes by CoryMartin-NOAA · Pull Request #2738 · NOAA-EMC/global-workflow

CoryMartin-NOAA · 2024-06-28T17:45:05Z

Description

This PR adds in support for computing files needed for the aerosol analysis B. This includes a new task, aeroanlgenb. This work was performed by both me and @andytangborn

Resolves #2501
Resolves #2737

~~Please note, I am still using python to create dictionaries for FileHandler. This will not be resolved in this PR but a future PR, an issue #2737 has been created to make note of this.~~
EDIT: this PR has been added to such that the above is no longer true

Type of change

New feature (adds functionality)

Change characteristics

Is this a breaking change (a change in existing functionality)? NO
Does this change require a documentation update? NO

How has this been tested?

Cycled test on Hera

Checklist

Any dependent changes have been merged and published
My code follows the style guidelines of this project
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
My changes generate no new warnings
New and existing tests pass with my changes
I have made corresponding changes to the documentation if necessary

…lobal-workflow into feature/aero-b

Added tasks to construct aero_diffparm.yaml from template.

emcbot · 2024-08-27T20:31:11Z

CI Update on Wcoss2 at 08/27/24 08:31:10 PM
============================================
Cloning and Building global-workflow PR: 2738
with PID: 15294 on host: clogin03

emcbot · 2024-08-27T21:19:49Z

Automated global-workflow Testing Results:

Machine: Wcoss2
Start: Tue Aug 27 20:41:27 UTC 2024 on clogin03
---------------------------------------------------
Build: Completed at 08/27/24 09:19:30 PM
Case setup: Completed for experiment C48_ATM_2d5b369f
Case setup: Skipped for experiment C48mx500_3DVarAOWCDA_2d5b369f
Case setup: Skipped for experiment C48_S2SWA_gefs_2d5b369f
Case setup: Completed for experiment C48_S2SW_2d5b369f
Case setup: Completed for experiment C96_atm3DVar_extended_2d5b369f
Case setup: Skipped for experiment C96_atm3DVar_2d5b369f
Case setup: Completed for experiment C96C48_hybatmaerosnowDA_2d5b369f
Case setup: Completed for experiment C96C48_hybatmDA_2d5b369f
Case setup: Skipped for experiment C96C48_ufs_hybatmDA_2d5b369f

WalterKolczynski-NOAA

Sorry, missed these the first time. Just method names, following the same reasoning as the other PR.

WalterKolczynski-NOAA · 2024-08-28T01:43:20Z

+        pass
+
+    @logit(logger)
+    def computeVariance(self) -> None:


Python style guidelines say method names should be snake case, not camel case.

Suggested change

def computeVariance(self) -> None:

def compute_variance(self) -> None:

WalterKolczynski-NOAA · 2024-08-28T01:43:32Z

+        pass
+
+    @logit(logger)
+    def computeDiffusion(self) -> None:


Suggested change

def computeDiffusion(self) -> None:

def compute_diffusion(self) -> None:

WalterKolczynski-NOAA · 2024-08-28T01:44:09Z

+        self.link_jediexe()
+
+    @logit(logger)
+    def interpBackground(self) -> None:


Suggested change

def interpBackground(self) -> None:

def interp_background(self) -> None:

WalterKolczynski-NOAA · 2024-08-28T01:48:46Z

-
    @logit(logger)
-    def execute(self: Analysis) -> None:
+    def variational(self: Analysis) -> None:


Best verb I could come up with here, open to others.

Suggested change

def variational(self: Analysis) -> None:

def var_assimilate(self: Analysis) -> None:

WalterKolczynski-NOAA · 2024-08-28T01:51:06Z

    # Instantiate the aerosol analysis task
    AeroAnl = AerosolAnalysis(config)
-    AeroAnl.execute()
+    AeroAnl.variational()


Suggested change

AeroAnl.variational()

AeroAnl.var_assimilate()

WalterKolczynski-NOAA · 2024-08-28T01:54:07Z

+    AeroB.interpBackground()
+    AeroB.computeVariance()
+    AeroB.computeDiffusion()


Suggested change

AeroB.interpBackground()

AeroB.computeVariance()

AeroB.computeDiffusion()

AeroB.interp_background()

AeroB.compute_variance()

AeroB.compute_diffusion()

WalterKolczynski-NOAA

Upon reflection, this PR has been hanging around long enough. Let's try to get it in as-is and we can address the method names later.

emcbot · 2024-08-28T08:49:15Z

All CI Test Cases Passed on Wcoss2:

Experiment C48_ATM_2d5b369f *** SUCCESS *** at 08/27/24 10:35:17 PM
Experiment C48_S2SW_2d5b369f *** SUCCESS *** at 08/27/24 10:49:10 PM
Experiment C96C48_hybatmDA_2d5b369f *** SUCCESS *** at 08/27/24 11:28:29 PM
Experiment C96C48_hybatmaerosnowDA_2d5b369f *** SUCCESS *** at 08/28/24 12:21:25 AM
Experiment C96_atm3DVar_extended_2d5b369f *** SUCCESS *** at 08/28/24 08:42:32 AM

WalterKolczynski-NOAA · 2024-08-28T19:37:08Z

@CoryMartin-NOAA Looks like merging Russ's PR created a conflict

CoryMartin-NOAA · 2024-08-28T19:39:12Z

@WalterKolczynski-NOAA resolved

emcbot · 2024-08-29T02:16:25Z

CI Passed on Hera in Build# 2
Built and ran in directory /scratch1/NCEPDEV/global/CI/2738


Experiment C48_ATM_40c8feb0 Completed 1 Cycles: *SUCCESS* at Wed Aug 28 23:23:56 UTC 2024
Experiment C48mx500_3DVarAOWCDA_40c8feb0 Completed 2 Cycles: *SUCCESS* at Wed Aug 28 23:42:33 UTC 2024
Experiment C96_atm3DVar_40c8feb0 Completed 3 Cycles: *SUCCESS* at Thu Aug 29 00:50:20 UTC 2024
Experiment C48_S2SWA_gefs_40c8feb0 Completed 1 Cycles: *SUCCESS* at Thu Aug 29 00:50:52 UTC 2024
Experiment C48_S2SW_40c8feb0 Completed 1 Cycles: *SUCCESS* at Thu Aug 29 01:02:52 UTC 2024
Experiment C96C48_hybatmDA_40c8feb0 Completed 3 Cycles: *SUCCESS* at Thu Aug 29 01:08:58 UTC 2024
Experiment C96C48_ufs_hybatmDA_40c8feb0 Completed 2 Cycles: *SUCCESS* at Thu Aug 29 01:21:22 UTC 2024
Experiment C96C48_hybatmaerosnowDA_40c8feb0 Completed 3 Cycles: *SUCCESS* at Thu Aug 29 02:16:17 UTC 2024

emcbot · 2024-08-29T20:04:02Z

CI Passed on Hercules in Build# 1
Built and ran in directory /work2/noaa/stmp/CI/HERCULES/2738


Experiment C48_ATM_40c8feb0 Completed 1 Cycles: *SUCCESS* at Thu Aug 29 14:03:04 CDT 2024
Experiment C48_S2SWA_gefs_40c8feb0 Completed 1 Cycles: *SUCCESS* at Thu Aug 29 14:58:13 CDT 2024
Experiment C48_S2SW_40c8feb0 Completed 1 Cycles: *SUCCESS* at Thu Aug 29 15:03:39 CDT 2024
Experiment C96_atm3DVar_40c8feb0 Completed 3 Cycles: *SUCCESS* at Thu Aug 29 15:03:41 CDT 2024
Experiment C96C48_hybatmDA_40c8feb0 Completed 3 Cycles: *SUCCESS* at Thu Aug 29 15:03:52 CDT 2024

CoryMartin-NOAA and others added 30 commits April 17, 2024 15:50

Change run to variational for aerosol analysis

8766c0d

More places for run to var

34eb98a

run to var with a find/replace

4bfd5e5

More changes needed; end of day commit

da333e9

Missed one

a6645a7

Save to start building and iterating on HPC

217db3b

update gdas submodule and link new exe

520b3ac

commit before starting to build test experiment

20885a5

make executable

e872517

Commit before some testing

5018801

update gdas hash

0cc3ada

commit after debugging and testing

6a24e30

norm

3df6b36

norm was not happy

cee3def

Relax Norm, it is Friday afternoon

69e9fa4

Merge branch 'develop' into feature/aero-b

03d7e54

more updates; looks like it is working

605fbff

Merge branch 'feature/aero-b' of https://github.com/corymartin-noaa/g…

315cf3d

…lobal-workflow into feature/aero-b

pynorms

67f4a28

Merge branch 'NOAA-EMC:develop' into feature/aero-b

b2fbb83

more fixes

406cc44

Merge branch 'feature/aero-b' of https://github.com/corymartin-noaa/g…

fd9dd8d

…lobal-workflow into feature/aero-b

Merge branch 'develop' into feature/aero-b

ebd3d8c

Added tasks to construct aero_diffparm.yaml from template.

e09656d

changed DIFF -> DIFFUSION and diff -> diffusion in files below

62457eb

Update parm/config/gfs/config.aeroanlgenb

17b6dcc

Merge pull request #3 from CoryMartin-NOAA/feature/diff_parm

a200b1e

Added tasks to construct aero_diffparm.yaml from template.

Merge branch 'develop' into feature/aero-b

ea8ace1

Additions for running diffusion operator.

5f32976

link gdas_fv3jedi_error_covariance_toolbox.x

228ed0a

emcbot added CI-Wcoss2-Running CI testing on WCOSS for this PR is in-progress and removed CI-Wcoss2-Building CI testing is cloning/building on WCOSS2 labels Aug 27, 2024

WalterKolczynski-NOAA requested changes Aug 28, 2024

View reviewed changes

WalterKolczynski-NOAA previously approved these changes Aug 28, 2024

View reviewed changes

emcbot added CI-Wcoss2-Passed CI testing on WCOSS for this PR has completed successfully and removed CI-Wcoss2-Running CI testing on WCOSS for this PR is in-progress labels Aug 28, 2024

Merge branch 'develop' into feature/aero-b

40c8feb

CoryMartin-NOAA dismissed WalterKolczynski-NOAA’s stale review via 40c8feb August 28, 2024 19:38

WalterKolczynski-NOAA added CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera CI-Hercules-Ready **CM use only** PR is ready for CI testing on Hercules labels Aug 28, 2024

Merge branch 'develop' into feature/aero-b

155fb5a

WalterKolczynski-NOAA approved these changes Aug 29, 2024

View reviewed changes

	def computeVariance(self) -> None:
	def compute_variance(self) -> None:

	def computeDiffusion(self) -> None:
	def compute_diffusion(self) -> None:

	def interpBackground(self) -> None:
	def interp_background(self) -> None:

	def variational(self: Analysis) -> None:
	def var_assimilate(self: Analysis) -> None:

Conversation

CoryMartin-NOAA commented Jun 28, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Type of change

Change characteristics

How has this been tested?

Checklist

Uh oh!

emcbot commented Aug 27, 2024

Uh oh!

emcbot commented Aug 27, 2024

Uh oh!

WalterKolczynski-NOAA left a comment

Choose a reason for hiding this comment

Uh oh!

WalterKolczynski-NOAA Aug 28, 2024

Choose a reason for hiding this comment

Uh oh!

WalterKolczynski-NOAA Aug 28, 2024

Choose a reason for hiding this comment

Uh oh!

WalterKolczynski-NOAA Aug 28, 2024

Choose a reason for hiding this comment

Uh oh!

WalterKolczynski-NOAA Aug 28, 2024

Choose a reason for hiding this comment

Uh oh!

WalterKolczynski-NOAA Aug 28, 2024

Choose a reason for hiding this comment

Uh oh!

WalterKolczynski-NOAA Aug 28, 2024

Choose a reason for hiding this comment

Uh oh!

WalterKolczynski-NOAA left a comment

Choose a reason for hiding this comment

Uh oh!

emcbot commented Aug 28, 2024

Uh oh!

WalterKolczynski-NOAA commented Aug 28, 2024

Uh oh!

CoryMartin-NOAA commented Aug 28, 2024

Uh oh!

emcbot commented Aug 29, 2024

Uh oh!

emcbot commented Aug 29, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

CoryMartin-NOAA commented Jun 28, 2024 •

edited

Loading