Skip to content

Simple Obs db#26

Merged
CoryMartin-NOAA merged 16 commits into
mainfrom
feature/obs_db
Mar 25, 2025
Merged

Simple Obs db#26
CoryMartin-NOAA merged 16 commits into
mainfrom
feature/obs_db

Conversation

@guillaumevernieres
Copy link
Copy Markdown
Contributor

@guillaumevernieres guillaumevernieres commented Mar 24, 2025

Comment thread ush/python/pygfs/obsdb/__init__.py Outdated
Comment thread ush/python/pygfs/obsdb/jrr_aod_db.py
@guillaumevernieres guillaumevernieres marked this pull request as ready for review March 24, 2025 21:42
Comment thread ush/python/pyobsforge/obsdb/obsdb.py Outdated
Comment thread ush/python/pyobsforge/obsdb/obsdb.py
@CoryMartin-NOAA CoryMartin-NOAA merged commit 594eeec into main Mar 25, 2025
@CoryMartin-NOAA CoryMartin-NOAA deleted the feature/obs_db branch March 25, 2025 16:37
CoryMartin-NOAA added a commit that referenced this pull request Oct 21, 2025
…sts (#12)

1. Simplified ctest structure.
My initial version is very stupid and inefficient!
e.g. initial ctest list for test_obsforge_satobs_satwnd_amv_seviri looks
like the following:
```
  Test  #38: test_obsforge_satobs_satwnd_amv_seviri_m8_bufr2netcdf
  Test  #39: test_obsforge_satobs_satwnd_amv_seviri_m8_bufr2netcdf_mpi4
  Test  #40: test_obsforge_satobs_satwnd_amv_seviri_m8_script2netcdf
  Test  #41: test_obsforge_satobs_satwnd_amv_seviri_m8_script2netcdf_mpi4
  Test  #42: test_obsforge_satobs_satwnd_amv_seviri_m8_bufr4backend
  Test  #43: test_obsforge_satobs_satwnd_amv_seviri_m8_bufr4backend_mpi4
  Test  #44: test_obsforge_satobs_satwnd_amv_seviri_m8_script4backend
  Test  #45: test_obsforge_satobs_satwnd_amv_seviri_m8_script4backend_mpi4
  Test  #46: test_obsforge_satobs_satwnd_amv_seviri_m9_bufr2netcdf
  Test  #47: test_obsforge_satobs_satwnd_amv_seviri_m9_bufr2netcdf_mpi4
  Test  #48: test_obsforge_satobs_satwnd_amv_seviri_m9_script2netcdf
  Test  #49: test_obsforge_satobs_satwnd_amv_seviri_m9_script2netcdf_mpi4
  Test  #50: test_obsforge_satobs_satwnd_amv_seviri_m9_bufr4backend
  Test  #51: test_obsforge_satobs_satwnd_amv_seviri_m9_bufr4backend_mpi4
  Test  #52: test_obsforge_satobs_satwnd_amv_seviri_m9_script4backend
  Test  #53: test_obsforge_satobs_satwnd_amv_seviri_m9_script4backend_mpi4
  Test  #54: test_obsforge_satobs_satwnd_amv_seviri_m10_bufr2netcdf
  Test  #55: test_obsforge_satobs_satwnd_amv_seviri_m10_bufr2netcdf_mpi4
  Test  #56: test_obsforge_satobs_satwnd_amv_seviri_m10_script2netcdf
  Test  #57: test_obsforge_satobs_satwnd_amv_seviri_m10_script2netcdf_mpi4
  Test  #58: test_obsforge_satobs_satwnd_amv_seviri_m10_bufr4backend
  Test  #59: test_obsforge_satobs_satwnd_amv_seviri_m10_bufr4backend_mpi4
  Test  #60: test_obsforge_satobs_satwnd_amv_seviri_m10_script4backend
  Test  #61: test_obsforge_satobs_satwnd_amv_seviri_m10_script4backend_mpi4
  Test  #62: test_obsforge_satobs_satwnd_amv_seviri_m11_bufr2netcdf
  Test  #63: test_obsforge_satobs_satwnd_amv_seviri_m11_bufr2netcdf_mpi4
  Test  #64: test_obsforge_satobs_satwnd_amv_seviri_m11_script2netcdf
  Test  #65: test_obsforge_satobs_satwnd_amv_seviri_m11_script2netcdf_mpi4
  Test  #66: test_obsforge_satobs_satwnd_amv_seviri_m11_bufr4backend
  Test  #67: test_obsforge_satobs_satwnd_amv_seviri_m11_bufr4backend_mpi4
  Test  #68: test_obsforge_satobs_satwnd_amv_seviri_m11_script4backend
  Test  #69: test_obsforge_satobs_satwnd_amv_seviri_m11_script4backend_mpi4
```

The simplified one looks like the following for satwnd_amv_seviri:
``` 
      Start 22: test_obsforge_satobs_satwnd_amv_seviri_bufr2netcdf
 7/38 Test #22: test_obsforge_satobs_satwnd_amv_seviri_bufr2netcdf ...........   Passed   16.69 sec
      Start 23: test_obsforge_satobs_satwnd_amv_seviri_bufr2netcdf_mpi4
 8/38 Test #23: test_obsforge_satobs_satwnd_amv_seviri_bufr2netcdf_mpi4 ......   Passed    9.47 sec
      Start 24: test_obsforge_satobs_satwnd_amv_seviri_script2netcdf
 9/38 Test #24: test_obsforge_satobs_satwnd_amv_seviri_script2netcdf .........   Passed   18.98 sec
      Start 25: test_obsforge_satobs_satwnd_amv_seviri_script2netcdf_mpi4
10/38 Test #25: test_obsforge_satobs_satwnd_amv_seviri_script2netcdf_mpi4 ....   Passed   13.31 sec
      Start 26: test_obsforge_satobs_satwnd_amv_seviri_bufr4backend
11/38 Test #26: test_obsforge_satobs_satwnd_amv_seviri_bufr4backend ..........   Passed   18.95 sec
      Start 27: test_obsforge_satobs_satwnd_amv_seviri_bufr4backend_mpi4
12/38 Test #27: test_obsforge_satobs_satwnd_amv_seviri_bufr4backend_mpi4 .....   Passed   11.18 sec
      Start 28: test_obsforge_satobs_satwnd_amv_seviri_script4backend
13/38 Test #28: test_obsforge_satobs_satwnd_amv_seviri_script4backend ........   Passed   22.38 sec
      Start 29: test_obsforge_satobs_satwnd_amv_seviri_script4backend_mpi4
14/38 Test #29: test_obsforge_satobs_satwnd_amv_seviri_script4backend_mpi4 ...   Passed   16.85 sec
```
The test structure is simplified in that the test for each data type,
and each configuration only needs to be tested once because the
`bufr_comp.sh` has been modified to handle comparisons of output files
from various satellite platforms for the same sensor type. (See
[bufr-query PR #53](NOAA-EMC/bufr-query#53))

2. Enable IASI tests
    - add tests in CMakeList.txt
    - add test references (testoutput) in obsForge-0.0.0.tgz on EMCRZDM
    - add test configuration YAMLs (testinput)
- mapping file and python configuration YAMLs are added in [SPOC
PR#25](NOAA-EMC/spoc#25)

---------

Co-authored-by: Cory Martin <cory.r.martin@noaa.gov>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement a Simple Database for Aero and Marine Obs

2 participants