Skip to content

saving original date + testing#24

Merged
guillaumevernieres merged 3 commits into
mainfrom
feature/origdate
Mar 25, 2025
Merged

saving original date + testing#24
guillaumevernieres merged 3 commits into
mainfrom
feature/origdate

Conversation

@givelberg
Copy link
Copy Markdown
Contributor

@givelberg givelberg commented Mar 23, 2025 •
edited by guillaumevernieres
Loading

Added originalDate and a corresponding test.
Changed optional int metadata to be int64 to be able to save dates.

guillaumevernieres
guillaumevernieres requested changes Mar 24, 2025
View reviewed changes
Copy link
Copy Markdown
Contributor

@guillaumevernieres guillaumevernieres left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changing "int" to "int64" will probably break the downstream applications.

Comment thread utils/preproc/NetCDFToIodaConverter.h
Comment thread utils/preproc/util.h
givelberg
givelberg commented Mar 25, 2025
View reviewed changes
Comment thread utils/preproc/util.h
guillaumevernieres
guillaumevernieres approved these changes Mar 25, 2025
View reviewed changes
Copy link
Copy Markdown
Contributor

@guillaumevernieres guillaumevernieres left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @givelberg

guillaumevernieres pushed a commit to NOAA-EMC/GDASApp that referenced this pull request Mar 25, 2025
@givelberg
added original date (#1570)
718c397 
# Description

# Companion PRs

<!-- NOAA-EMC/obsForge#24 -->

# Issues
NOAA-EMC/obsForge#23
Resolves #23

# Automated CI tests to run in Global Workflow
<!-- Which Global Workflow CI tests are required to adequately test this
PR? -->
- [ ] atm_jjob <!-- JEDI atm single cycle DA !-->
- [ ] C96C48_ufs_hybatmDA <!-- JEDI atm cycled DA !-->
- [ ] C96C48_hybatmaerosnowDA  <!-- JEDI aero/snow cycled DA !-->
- [ ] C48mx500_3DVarAOWCDA <!-- JEDI low-res marine 3DVar cycled DA !-->
- [x] C48mx500_hybAOWCDA <!-- JEDI marine hybrid envar cycled DA !-->
- [ ] C96C48_hybatmDA <!-- GSI atm cycled DA !-->

Co-authored-by: Edward Givelberg <edwardg@hercules-login-4.hpc.msstate.edu>
@guillaumevernieres guillaumevernieres merged commit 5a2c1f9 into main Mar 25, 2025
@guillaumevernieres guillaumevernieres deleted the feature/origdate branch March 25, 2025 23:13
jspsl pushed a commit to NOAA-EMC/GDASApp that referenced this pull request Mar 26, 2025
@givelberg @jspsl
added original date (#1570)
14dbdce 
# Description

# Companion PRs

<!-- NOAA-EMC/obsForge#24 -->

# Issues
NOAA-EMC/obsForge#23
Resolves #23

# Automated CI tests to run in Global Workflow
<!-- Which Global Workflow CI tests are required to adequately test this
PR? -->
- [ ] atm_jjob <!-- JEDI atm single cycle DA !-->
- [ ] C96C48_ufs_hybatmDA <!-- JEDI atm cycled DA !-->
- [ ] C96C48_hybatmaerosnowDA  <!-- JEDI aero/snow cycled DA !-->
- [ ] C48mx500_3DVarAOWCDA <!-- JEDI low-res marine 3DVar cycled DA !-->
- [x] C48mx500_hybAOWCDA <!-- JEDI marine hybrid envar cycled DA !-->
- [ ] C96C48_hybatmDA <!-- GSI atm cycled DA !-->

Co-authored-by: Edward Givelberg <edwardg@hercules-login-4.hpc.msstate.edu>
CoryMartin-NOAA added a commit that referenced this pull request Oct 21, 2025
@emilyhcliu @CoryMartin-NOAA
Simplified ctest structure for satellite observations and add IASI te…
0b1c281 
…sts (#12)

1. Simplified ctest structure.
My initial version is very stupid and inefficient!
e.g. initial ctest list for test_obsforge_satobs_satwnd_amv_seviri looks
like the following:
```
  Test  #38: test_obsforge_satobs_satwnd_amv_seviri_m8_bufr2netcdf
  Test  #39: test_obsforge_satobs_satwnd_amv_seviri_m8_bufr2netcdf_mpi4
  Test  #40: test_obsforge_satobs_satwnd_amv_seviri_m8_script2netcdf
  Test  #41: test_obsforge_satobs_satwnd_amv_seviri_m8_script2netcdf_mpi4
  Test  #42: test_obsforge_satobs_satwnd_amv_seviri_m8_bufr4backend
  Test  #43: test_obsforge_satobs_satwnd_amv_seviri_m8_bufr4backend_mpi4
  Test  #44: test_obsforge_satobs_satwnd_amv_seviri_m8_script4backend
  Test  #45: test_obsforge_satobs_satwnd_amv_seviri_m8_script4backend_mpi4
  Test  #46: test_obsforge_satobs_satwnd_amv_seviri_m9_bufr2netcdf
  Test  #47: test_obsforge_satobs_satwnd_amv_seviri_m9_bufr2netcdf_mpi4
  Test  #48: test_obsforge_satobs_satwnd_amv_seviri_m9_script2netcdf
  Test  #49: test_obsforge_satobs_satwnd_amv_seviri_m9_script2netcdf_mpi4
  Test  #50: test_obsforge_satobs_satwnd_amv_seviri_m9_bufr4backend
  Test  #51: test_obsforge_satobs_satwnd_amv_seviri_m9_bufr4backend_mpi4
  Test  #52: test_obsforge_satobs_satwnd_amv_seviri_m9_script4backend
  Test  #53: test_obsforge_satobs_satwnd_amv_seviri_m9_script4backend_mpi4
  Test  #54: test_obsforge_satobs_satwnd_amv_seviri_m10_bufr2netcdf
  Test  #55: test_obsforge_satobs_satwnd_amv_seviri_m10_bufr2netcdf_mpi4
  Test  #56: test_obsforge_satobs_satwnd_amv_seviri_m10_script2netcdf
  Test  #57: test_obsforge_satobs_satwnd_amv_seviri_m10_script2netcdf_mpi4
  Test  #58: test_obsforge_satobs_satwnd_amv_seviri_m10_bufr4backend
  Test  #59: test_obsforge_satobs_satwnd_amv_seviri_m10_bufr4backend_mpi4
  Test  #60: test_obsforge_satobs_satwnd_amv_seviri_m10_script4backend
  Test  #61: test_obsforge_satobs_satwnd_amv_seviri_m10_script4backend_mpi4
  Test  #62: test_obsforge_satobs_satwnd_amv_seviri_m11_bufr2netcdf
  Test  #63: test_obsforge_satobs_satwnd_amv_seviri_m11_bufr2netcdf_mpi4
  Test  #64: test_obsforge_satobs_satwnd_amv_seviri_m11_script2netcdf
  Test  #65: test_obsforge_satobs_satwnd_amv_seviri_m11_script2netcdf_mpi4
  Test  #66: test_obsforge_satobs_satwnd_amv_seviri_m11_bufr4backend
  Test  #67: test_obsforge_satobs_satwnd_amv_seviri_m11_bufr4backend_mpi4
  Test  #68: test_obsforge_satobs_satwnd_amv_seviri_m11_script4backend
  Test  #69: test_obsforge_satobs_satwnd_amv_seviri_m11_script4backend_mpi4
```

The simplified one looks like the following for satwnd_amv_seviri:
``` 
      Start 22: test_obsforge_satobs_satwnd_amv_seviri_bufr2netcdf
 7/38 Test #22: test_obsforge_satobs_satwnd_amv_seviri_bufr2netcdf ...........   Passed   16.69 sec
      Start 23: test_obsforge_satobs_satwnd_amv_seviri_bufr2netcdf_mpi4
 8/38 Test #23: test_obsforge_satobs_satwnd_amv_seviri_bufr2netcdf_mpi4 ......   Passed    9.47 sec
      Start 24: test_obsforge_satobs_satwnd_amv_seviri_script2netcdf
 9/38 Test #24: test_obsforge_satobs_satwnd_amv_seviri_script2netcdf .........   Passed   18.98 sec
      Start 25: test_obsforge_satobs_satwnd_amv_seviri_script2netcdf_mpi4
10/38 Test #25: test_obsforge_satobs_satwnd_amv_seviri_script2netcdf_mpi4 ....   Passed   13.31 sec
      Start 26: test_obsforge_satobs_satwnd_amv_seviri_bufr4backend
11/38 Test #26: test_obsforge_satobs_satwnd_amv_seviri_bufr4backend ..........   Passed   18.95 sec
      Start 27: test_obsforge_satobs_satwnd_amv_seviri_bufr4backend_mpi4
12/38 Test #27: test_obsforge_satobs_satwnd_amv_seviri_bufr4backend_mpi4 .....   Passed   11.18 sec
      Start 28: test_obsforge_satobs_satwnd_amv_seviri_script4backend
13/38 Test #28: test_obsforge_satobs_satwnd_amv_seviri_script4backend ........   Passed   22.38 sec
      Start 29: test_obsforge_satobs_satwnd_amv_seviri_script4backend_mpi4
14/38 Test #29: test_obsforge_satobs_satwnd_amv_seviri_script4backend_mpi4 ...   Passed   16.85 sec
```
The test structure is simplified in that the test for each data type,
and each configuration only needs to be tested once because the
`bufr_comp.sh` has been modified to handle comparisons of output files
from various satellite platforms for the same sensor type. (See
[bufr-query PR #53](NOAA-EMC/bufr-query#53))

2. Enable IASI tests
    - add tests in CMakeList.txt
    - add test references (testoutput) in obsForge-0.0.0.tgz on EMCRZDM
    - add test configuration YAMLs (testinput)
- mapping file and python configuration YAMLs are added in [SPOC
PR#25](NOAA-EMC/spoc#25)

---------

Co-authored-by: Cory Martin <cory.r.martin@noaa.gov>
DavidNew-NOAA pushed a commit to NOAA-EMC/GDASApp that referenced this pull request Jan 16, 2026
@givelberg
added original date (#1570)
11304d5 
# Description

# Companion PRs

<!-- NOAA-EMC/obsForge#24 -->

# Issues
NOAA-EMC/obsForge#23
Resolves #23

# Automated CI tests to run in Global Workflow
<!-- Which Global Workflow CI tests are required to adequately test this
PR? -->
- [ ] atm_jjob <!-- JEDI atm single cycle DA !-->
- [ ] C96C48_ufs_hybatmDA <!-- JEDI atm cycled DA !-->
- [ ] C96C48_hybatmaerosnowDA  <!-- JEDI aero/snow cycled DA !-->
- [ ] C48mx500_3DVarAOWCDA <!-- JEDI low-res marine 3DVar cycled DA !-->
- [x] C48mx500_hybAOWCDA <!-- JEDI marine hybrid envar cycled DA !-->
- [ ] C96C48_hybatmDA <!-- GSI atm cycled DA !-->

Co-authored-by: Edward Givelberg <edwardg@hercules-login-4.hpc.msstate.edu>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@guillaumevernieres guillaumevernieres guillaumevernieres approved these changes

@ilianagenkova ilianagenkova Awaiting requested review from ilianagenkova

@apchoiCMD apchoiCMD Awaiting requested review from apchoiCMD

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Keep the original date when "redating"

2 participants

@givelberg @guillaumevernieres