Skip to content

Remove tests with R2D2 and anything referencing it#836

Merged
guillaumevernieres merged 28 commits into
developfrom
feature/remove-r2d2
Jan 10, 2024
Merged

Remove tests with R2D2 and anything referencing it#836
guillaumevernieres merged 28 commits into
developfrom
feature/remove-r2d2

Conversation

@AndrewEichmann-NOAA
Copy link
Copy Markdown
Collaborator

@AndrewEichmann-NOAA AndrewEichmann-NOAA commented Jan 4, 2024

Removes soca ctests related to R2D2, associated data, and any lines in scripts, yamls, or other config files referencing it. SOCA ctests now run on obs prepped by the prep obs task.

Almost takes care of #815 as references are still present in global-workflow task config files.

logging.info has been replaced by print because logging hasn't been working

@AndrewEichmann-NOAA AndrewEichmann-NOAA marked this pull request as ready for review January 4, 2024 21:31
@AndrewEichmann-NOAA AndrewEichmann-NOAA added the hera-GW-RT Queue for automated testing with global-workflow on Hera label Jan 4, 2024
@AndrewEichmann-NOAA
Copy link
Copy Markdown
Collaborator Author

AndrewEichmann-NOAA commented Jan 5, 2024

Can we remove it here:

- name: Install other dependencies

@CoryMartin-NOAA The whole - name: section, or just the three lines with "r2d2"?

@CoryMartin-NOAA
Copy link
Copy Markdown
Contributor

Do we have any need for solo ? I don't think so, right?

ShastriPaturi
ShastriPaturi previously approved these changes Jan 5, 2024
Copy link
Copy Markdown
Collaborator

@ShastriPaturi ShastriPaturi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will test with my in situ obs.
Approving it for now.

@guillaumevernieres
Copy link
Copy Markdown
Contributor

Do we have any need for solo ? I don't think so, right?

no need for solo anymore @CoryMartin-NOAA .

@guillaumevernieres
Copy link
Copy Markdown
Contributor

Can we remove it here:

- name: Install other dependencies

@CoryMartin-NOAA The whole - name: section, or just the three lines with "r2d2"?

The entire section @AndrewEichmann-NOAA ... I'm 99.3% sure we don't use solo outside of the r2d2 implementation ...

@AndrewEichmann-NOAA
Copy link
Copy Markdown
Collaborator Author

@CoryMartin-NOAA @guillaumevernieres ok, removed, let's see what happens

@AndrewEichmann-NOAA AndrewEichmann-NOAA added hera-GW-RT Queue for automated testing with global-workflow on Hera and removed hera-GW-RT-Failed Automated testing with global-workflow failed on Hera labels Jan 5, 2024
@emcbot emcbot added hera-GW-RT-Running Automated testing with global-workflow running on Hera and removed hera-GW-RT Queue for automated testing with global-workflow on Hera labels Jan 5, 2024
@emcbot
Copy link
Copy Markdown

emcbot commented Jan 5, 2024

Automated Global-Workflow GDASApp Testing Results:
Machine: hera

Start: Fri Jan  5 22:00:21 UTC 2024 on hfe07
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Fri Jan  5 22:14:20 UTC 2024
---------------------------------------------------
Tests:                                  *Failed*
Tests: Failed at Fri Jan  5 23:28:27 UTC 2024
Tests: 94% tests passed, 3 tests failed out of 52
	1688 - test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_VRFY (Failed)
	1705 - test_gdasapp_atm_jjob_ens_final (Failed)
Tests: see output at /scratch1/NCEPDEV/da/Cory.R.Martin/CI/GDASApp/workflow/PR/836/global-workflow/sorc/gdas.cd/build/log.ctest

@emcbot emcbot added hera-GW-RT-Failed Automated testing with global-workflow failed on Hera and removed hera-GW-RT-Running Automated testing with global-workflow running on Hera labels Jan 5, 2024
@AndrewEichmann-NOAA
Copy link
Copy Markdown
Collaborator Author

test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_VRFY fails, test_gdasapp_atm_jjob_ens_run times out, and test_gdasapp_atm_jjob_ens_final fails - the vrfy is a known issue, what of the other two?

@guillaumevernieres
Copy link
Copy Markdown
Contributor

test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_VRFY fails, test_gdasapp_atm_jjob_ens_run times out, and test_gdasapp_atm_jjob_ens_final fails - the vrfy is a known issue, what of the other two?

I suspect the test_gdasapp_atm_jjob_ens_final fails because of test_gdasapp_atm_jjob_ens_run and is probably unrelated to your changes ... So maybe we can ignore? @RussTreadon-NOAA , @CoryMartin-NOAA ?

Copy link
Copy Markdown
Contributor

@guillaumevernieres guillaumevernieres left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good @AndrewEichmann-NOAA , do we have the dependencies sorted out properly in the g-w? If not it needs to be done asap, since merging this would break the cycling.

@RussTreadon-NOAA
Copy link
Copy Markdown
Contributor

Yes, @guillaumevernieres, you are correct. test_gdasapp_atm_jjob_ens_final processes output from test_gdasapp_atm_jjob_ens_run. If the ens or var run job fails, the corresponding final job will fail. The ens_run job executes fv3jedi_letkf.x with 36 tasks. Depending on the system load, the queue wait time can exceed the ctest timeout limit.

@AndrewEichmann-NOAA
Copy link
Copy Markdown
Collaborator Author

@guillaumevernieres The changes to g-w depend on this PR and I don't think vice versa, so once this is in I can change g-w. I think it's just a matter of removing assignments from config files.

Copy link
Copy Markdown
Contributor

@guillaumevernieres guillaumevernieres left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving but lets not merge just quite yet until we have an understanding of how close we are with the corresponding g-w updates.
Thanks for doing this @AndrewEichmann-NOAA .

@guillaumevernieres guillaumevernieres merged commit 6577584 into develop Jan 10, 2024
@CoryMartin-NOAA CoryMartin-NOAA deleted the feature/remove-r2d2 branch March 28, 2024 15:24
guillaumevernieres pushed a commit that referenced this pull request Apr 18, 2024
We removed these from the unit tests in #836 but missed this one. This
is failing because of our RZDM issues, but it exposed that these need to
be removed.
DavidNew-NOAA pushed a commit that referenced this pull request Jan 16, 2026
Removes soca ctests related to R2D2, associated data, and any lines in
scripts, yamls, or other config files referencing it. SOCA ctests now
run on obs prepped by the prep obs task.

Almost takes care of #815 as
references are still present in global-workflow task config files.

`logging.info` has been replaced by `print` because logging hasn't been
working

---------

Co-authored-by: Guillaume Vernieres <guillaume.vernieres@noaa.gov>
DavidNew-NOAA pushed a commit that referenced this pull request Jan 16, 2026
We removed these from the unit tests in #836 but missed this one. This
is failing because of our RZDM issues, but it exposed that these need to
be removed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

hera-GW-RT-Failed Automated testing with global-workflow failed on Hera

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants