Skip to content

Add PlanetiQ YAM8 SAID=768 RO data in GSI v16#930

Closed
XuanliLi-NOAA wants to merge 1 commit into
NOAA-EMC:develop-v16from
XuanliLi-NOAA:v16_piqyam8
Closed

Add PlanetiQ YAM8 SAID=768 RO data in GSI v16#930
XuanliLi-NOAA wants to merge 1 commit into
NOAA-EMC:develop-v16from
XuanliLi-NOAA:v16_piqyam8

Conversation

@XuanliLi-NOAA

@XuanliLi-NOAA XuanliLi-NOAA commented Sep 11, 2025

Copy link
Copy Markdown
Contributor

Description

This PR addresses issue #928 by adding satellite ID 768 commercial PlanetiQ YAM8 RO data to GSI v16. setupbend.f90 has been updated to apply the commercial RO QC filters for this data, and the print statement for the number of RO obs considered and accepted has been turned off. Note that the fix submodule in this fork is the develop branch, and it works fine for the GSI sample data test runs.

Resolves #928.

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

How Has This Been Tested?

A GSI test was conducted on WCOSS2 using sample PlanetiQ YAM8 files with SAID = 768 provided by UCAR. Additional testing was performed with files containing PlanetiQ data from both SAID 267 (GNOMES satellites) and 768. Both tests successfully assimilated the YAM8 data, and the statistics in fort.212 look good for PlanetiQ.

A GSI regression test against the develop-v16 branch (262ce16) was run on WCOSS2. All tests passed.
Test project /lfs/h2/emc/da/noscrub/xuanli.li/git/v16_piqyam8_768/GSI/build
Start 1: global_4denvar
Start 2: rtma
Start 3: rrfs_3denvar_rdasens
Start 4: hafs_4denvar_glbens
Start 5: hafs_3denvar_hybens
Start 6: global_enkf
1/6 Test #3: rrfs_3denvar_rdasens ............. Passed 736.16 sec
2/6 Test #2: rtma ............................. Passed 1157.18 sec
3/6 Test #5: hafs_3denvar_hybens .............. Passed 1285.03 sec
4/6 Test #4: hafs_4denvar_glbens .............. Passed 1534.61 sec
5/6 Test #6: global_enkf ...................... Passed 1536.60 sec
6/6 Test #1: global_4denvar ................... Passed 1985.11 sec

100% tests passed, 0 tests failed out of 6

Total Test time (real) = 1985.12 sec

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • New and existing tests pass with my changes
  • Any dependent changes have been merged and published

@XuanliLi-NOAA

Copy link
Copy Markdown
Contributor Author

@RussTreadon-NOAA Could you please add @ADCollard and @emilyhcliu as the reviewers? Thank you!

@RussTreadon-NOAA

Copy link
Copy Markdown
Contributor

@XuanliLi-NOAA: It's my understanding that this PR does not need to be merged into develop-v16 ASAP. If true, would you please change this PR to draft?

@RussTreadon-NOAA

Copy link
Copy Markdown
Contributor

@XuanliLi-NOAA : What is the status of this draft PR?

@XuanliLi-NOAA

Copy link
Copy Markdown
Contributor Author

@RussTreadon-NOAA We have completed two 2-week parallel experiments — nopiq and allpiq. The results confirm that temperature and height biases against radiosonde data increase with the number of RO observations assimilated. Therefore, @danholdaway suggested turning on the PlanetiQ YAM-8 data later (together with the MSG data), and @ADCollard confirmed this will likely happen sometime early next year.

In the meantime, we are running sensitivity experiments with different k1 coefficients of the obs operator. Preliminary results show a reduction in the biases, and we hope this approach may help address the forecast bias issue.

@RussTreadon-NOAA

Copy link
Copy Markdown
Contributor

@XuanliLi-NOAA : Thank you for the update. Given what you shared does it make sense to close this draft PR and open a new PR at a later date?

@XuanliLi-NOAA

Copy link
Copy Markdown
Contributor Author

Yes, that would be fine. Thank you!

@RussTreadon-NOAA

Copy link
Copy Markdown
Contributor

Close this PR as development continue. A new PR will be opened at a later date.

@RussTreadon-NOAA

Copy link
Copy Markdown
Contributor

Reopen PR as per remarks in issue #928.

@XuanliLi-NOAA XuanliLi-NOAA closed this by deleting the head repository Dec 8, 2025
@XuanliLi-NOAA

Copy link
Copy Markdown
Contributor Author

@RussTreadon-NOAA and @ADCollard The ctest for develop-v16 on WCOSS2 failed for global_enkf:
Test project /lfs/h2/emc/da/noscrub/xuanli.li/git/v16_768/gsi/GSI/build
Start 1: global_4denvar
Start 2: rtma
Start 3: rrfs_3denvar_rdasens
Start 4: hafs_4denvar_glbens
Start 5: hafs_3denvar_hybens
Start 6: global_enkf
1/6 Test #6: global_enkf ......................***Failed 120.76 sec
2/6 Test #3: rrfs_3denvar_rdasens ............. Passed 738.37 sec
3/6 Test #2: rtma ............................. Passed 1171.43 sec
4/6 Test #5: hafs_3denvar_hybens .............. Passed 1252.65 sec
5/6 Test #4: hafs_4denvar_glbens .............. Passed 1529.23 sec
6/6 Test #1: global_4denvar ................... Passed 1985.30 sec
83% tests passed, 1 tests failed out of 6
The following tests FAILED:
6 - global_enkf (Failed)

The stderr in global_enkf_loproc_updat shows the following messages:
2 No such file or directory
/lfs/h2/emc/ptmp/xuanli.li/GSI/tmpreg_global_enkf/global_enkf_loproc_updat/bfg_2024022300_fhr03_mem007
99
2 No such file or directory
/lfs/h2/emc/ptmp/xuanli.li/GSI/tmpreg_global_enkf/global_enkf_loproc_updat/bfg_ 2024022300_fhr03_mem007
99
11 Resource temporarily unavailable
/lfs/h2/emc/ptmp/xuanli.li/GSI/tmpreg_global_enkf/global_enkf_loproc_updat/bfg_ 2024022300_fhr03_mem007
99
nid001126.cactus.wcoss2.ncep.noaa.gov: rank 6 exited with code 99
forrtl: error (78): process killed (SIGTERM)
Image PC Routine Line Source
enkf.x 0000000001AD25EB Unknown Unknown Unknown
libpthread-2.31.s 0000150843CD0910 Unknown Unknown Unknown
libmpi_intel.so.1 000015083658DD46 Unknown Unknown Unknown
libmpi_intel.so.1 0000150836629245 Unknown Unknown Unknown
libmpi_intel.so.1 0000150834E34D3E PMPI_Barrier Unknown Unknown
libmpifort_intel. 000015083AC45FCB pmpi_barrier Unknown Unknown
enkf.x 00000000004C5320 gridio_mp_readgri 482 gridio_gfs.f90
enkf.x 00000000004B1B95 statevec_mp_read_ 191 statevec.f90
enkf.x 0000000000413605 MAIN__ 150 enkf_main.f90

Do you have any suggestions on how to solve this? Thank you!

@RussTreadon-NOAA

Copy link
Copy Markdown
Contributor

Your /lfs/h2/emc/da/noscrub/xuanli.li/git/v16_768/gsi/GSI/fix is GSI-fix develop at 20b4748. This is the current head of GSI-fix develop. This is not the correct fix to use with v16 GSI.

Your fix/global_anavinfo.l127.txt contains

  t2m       1      -1         2m_temperature       tmp2m
  q2m       1      -1         2m_spec_humidity     spfh2m

in the met_guess section. These variables are in the surface file (bfg_*). The v16 regression/global_enkf.sh does not include copies for the surface flies. The develop regression/global_enkf.sh includes these copies.

Your run fails because it's trying to read in data for these 2m fields from the surface file but the surface file is not present. This is the error message your quote

2 No such file or directory
 /lfs/h2/emc/ptmp/xuanli.li/GSI/tmpreg_global_enkf/global_enkf_loproc_updat/bfg_2024022300_fhr03_mem007

GSI develop-v16 has fix @ 3c98350. The global_anavinfo.l127.txt with this hash does not contain 2m variables.

I can't find your branch XuanliLi-NOAA:v16_piqyam8 in github to see what fix it points at.

@XuanliLi-NOAA

Copy link
Copy Markdown
Contributor Author

@RussTreadon-NOAA Thank you so much for finding that! I'll use the correct gsi-fix and see how it goes.

@XuanliLi-NOAA

Copy link
Copy Markdown
Contributor Author

Thank you @RussTreadon-NOAA! I changed the branch of fix from develop to 3c98350. Now all 6 tests passed. I'll create a new PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants