Skip to content

Refactoring prepocnobs task#1049

Merged
guillaumevernieres merged 18 commits into
developfrom
feature/refactor-prepobs
Apr 19, 2024
Merged

Refactoring prepocnobs task#1049
guillaumevernieres merged 18 commits into
developfrom
feature/refactor-prepobs

Conversation

@AndrewEichmann-NOAA
Copy link
Copy Markdown
Collaborator

@AndrewEichmann-NOAA AndrewEichmann-NOAA commented Apr 16, 2024

Makes prepocnobs a wxflow Task. Also tweaks the BUFR to IODA converter for argo floats, necessary for testing purposes.

Fixes #1047

apchoiCMD
apchoiCMD previously approved these changes Apr 16, 2024
View reviewed changes
Copy link
Copy Markdown
Collaborator

@apchoiCMD apchoiCMD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Look good-

@guillaumevernieres guillaumevernieres added orion-GW-RT Queue for automated testing with global-workflow on Orion hera-GW-RT Queue for automated testing with global-workflow on Hera labels Apr 16, 2024
@emcbot emcbot added hera-GW-RT-Running Automated testing with global-workflow running on Hera orion-GW-RT-Running Automated testing with global-workflow running on Orion and removed hera-GW-RT Queue for automated testing with global-workflow on Hera orion-GW-RT Queue for automated testing with global-workflow on Orion labels Apr 16, 2024
@emcbot
Copy link
Copy Markdown

emcbot commented Apr 17, 2024

Automated Global-Workflow GDASApp Testing Results:
Machine: hera

Start: Wed Apr 17 00:35:42 UTC 2024 on hfe06
---------------------------------------------------
Tests:                                  *Failed*
Tests: Failed at Wed Apr 17 00:36:27 UTC 2024
Tests: 
Tests: see output at /scratch1/NCEPDEV/da/Cory.R.Martin/CI/GDASApp/workflow/PR/1049/global-workflow/sorc/gdas.cd/build/log.ctest

@emcbot emcbot added hera-GW-RT-Failed Automated testing with global-workflow failed on Hera and removed hera-GW-RT-Running Automated testing with global-workflow running on Hera labels Apr 17, 2024
@emcbot
Copy link
Copy Markdown

emcbot commented Apr 17, 2024

Automated Global-Workflow GDASApp Testing Results:
Machine: orion

Start: Tue Apr 16 19:45:36 CDT 2024 on Orion-login-1.HPC.MsState.Edu
---------------------------------------------------
Tests:                                  *Failed*
Tests: Failed at Tue Apr 16 19:46:25 CDT 2024
Tests: 
Tests: see output at /work2/noaa/stmp/cmartin/CI/GDASApp/workflow/PR/1049/global-workflow/sorc/gdas.cd/build/log.ctest

@emcbot emcbot added orion-GW-RT-Failed Automated testing with global-workflow failed on Orion and removed orion-GW-RT-Running Automated testing with global-workflow running on Orion labels Apr 17, 2024
@emcbot
Copy link
Copy Markdown

emcbot commented Apr 17, 2024

Automated Global-Workflow GDASApp Testing Results:
Machine: hera

Start: Wed Apr 17 00:35:42 UTC 2024 on hfe06
---------------------------------------------------
Tests:                                  *Failed*
Tests: Failed at Wed Apr 17 00:36:27 UTC 2024
Tests: 
Tests: see output at /scratch1/NCEPDEV/da/Cory.R.Martin/CI/GDASApp/workflow/PR/1049/global-workflow/sorc/gdas.cd/build/log.ctest

Build: SUCCESS
Build: Completed at Wed Apr 17 01:27:04 UTC 2024

Tests: Failed
Tests: Failed at Wed Apr 17 01:42:14 UTC 2024
Tests: 91% tests passed, 5 tests failed out of 54
1763 - test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_VRFY (Failed)
1776 - test_gdasapp_atm_jjob_var_run (Failed)
1777 - test_gdasapp_atm_jjob_var_inc (Failed)
1778 - test_gdasapp_atm_jjob_var_final (Failed)
Tests: see output at /scratch1/NCEPDEV/da/Cory.R.Martin/CI/GDASApp/workflow/PR/1049/global-workflow/sorc/gdas.cd/build/log.ctest

@emcbot
Copy link
Copy Markdown

emcbot commented Apr 17, 2024

Automated Global-Workflow GDASApp Testing Results:
Machine: orion

Start: Tue Apr 16 19:45:36 CDT 2024 on Orion-login-1.HPC.MsState.Edu
---------------------------------------------------
Tests:                                  *Failed*
Tests: Failed at Tue Apr 16 19:46:25 CDT 2024
Tests: 
Tests: see output at /work2/noaa/stmp/cmartin/CI/GDASApp/workflow/PR/1049/global-workflow/sorc/gdas.cd/build/log.ctest

Build: SUCCESS
Build: Completed at Tue Apr 16 20:40:48 CDT 2024

Tests: Failed
Tests: Failed at Tue Apr 16 21:03:15 CDT 2024
Tests: 93% tests passed, 4 tests failed out of 54
1777 - test_gdasapp_atm_jjob_var_run (Failed)
1778 - test_gdasapp_atm_jjob_var_inc (Failed)
1779 - test_gdasapp_atm_jjob_var_final (Failed)
Tests: see output at /work2/noaa/stmp/cmartin/CI/GDASApp/workflow/PR/1049/global-workflow/sorc/gdas.cd/build/log.ctest

guillaumevernieres
guillaumevernieres reviewed Apr 17, 2024
View reviewed changes
Copy link
Copy Markdown
Contributor

@guillaumevernieres guillaumevernieres left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice and clean! Thanks @AndrewEichmann-NOAA
Looks like we need to do a short test run with all/most of the obs types before we merge this.

Comment thread ush/soca/prep_ocean_obs.py Outdated
@AndrewEichmann-NOAA AndrewEichmann-NOAA marked this pull request as ready for review April 17, 2024 13:05
@AndrewEichmann-NOAA
Copy link
Copy Markdown
Collaborator Author

AndrewEichmann-NOAA commented Apr 17, 2024

Nice and clean! Thanks @AndrewEichmann-NOAA Looks like we need to do a short test run with all/most of the obs types before we merge this.

@guillaumevernieres What's the latest in experiment configs to do this?

CoryMartin-NOAA
CoryMartin-NOAA reviewed Apr 18, 2024
View reviewed changes
Comment thread ush/soca/prep_ocean_obs.py Outdated
Comment thread ush/soca/prep_ocean_obs.py Outdated
Comment thread ush/soca/prep_ocean_obs_utils.py Outdated
@AndrewEichmann-NOAA
Copy link
Copy Markdown
Collaborator Author

AndrewEichmann-NOAA commented Apr 18, 2024

PYTHONPATH fix got merged: NOAA-EMC/global-workflow#2504

@guillaumevernieres guillaumevernieres merged commit 3ecda43 into develop Apr 19, 2024
@guillaumevernieres guillaumevernieres deleted the feature/refactor-prepobs branch April 19, 2024 15:28
@AndrewEichmann-NOAA AndrewEichmann-NOAA added hera-GW-RT Queue for automated testing with global-workflow on Hera orion-GW-RT Queue for automated testing with global-workflow on Orion and removed hera-GW-RT-Failed Automated testing with global-workflow failed on Hera orion-GW-RT-Failed Automated testing with global-workflow failed on Orion labels Apr 19, 2024
danholdaway added a commit that referenced this pull request Apr 23, 2024
@danholdaway
Merge remote-tracking branch 'upstream/develop' into feature/use_jcb_atm
b0591dd 
* upstream/develop:
  More marine DA bufixes  (#1069)
  Add basin flags to the ocean obs (#1067)
  Revert "Prepare observations for snow DA updates to the ensemble members" (#1066)
  Fix aero_diagb template (#1064)
  Refactoring prepocnobs task (#1049)
  Ensemble diagnostics not triggered properly (#1060)
  New ocean/ice B diag (#1050)
  Remove R2D2 and SOLO from the GitHub Actions workflow CI (#1058)
  Variance Partition estimates of Stddev for aerosols (#1037)
  Removing ctests of an obsolete bufr_converter in SOCA (#1056)
DavidNew-NOAA pushed a commit that referenced this pull request Jan 16, 2026
@AndrewEichmann-NOAA @guillaumevernieres @CoryMartin-NOAA
Refactoring prepocnobs task (#1049)
5c536e3 
Makes prepocnobs a wxflow Task. Also tweaks the BUFR to IODA converter
for argo floats, necessary for testing purposes.

Fixes #1047

---------

Co-authored-by: Guillaume Vernieres <guillaume.vernieres@noaa.gov>
Co-authored-by: Cory Martin <cory.r.martin@noaa.gov>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@CoryMartin-NOAA CoryMartin-NOAA CoryMartin-NOAA approved these changes

@guillaumevernieres guillaumevernieres Awaiting requested review from guillaumevernieres

@apchoiCMD apchoiCMD Awaiting requested review from apchoiCMD

Assignees

No one assigned

Labels

hera-GW-RT Queue for automated testing with global-workflow on Hera orion-GW-RT Queue for automated testing with global-workflow on Orion

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

JGLOBAL_PREP_OCEAN_OBS, should still be a stand alone job but its ex-script needs to be following what was done for the other DA applications

5 participants

@AndrewEichmann-NOAA @emcbot @CoryMartin-NOAA @guillaumevernieres @apchoiCMD