add PlanetiQ YAM-8 data with SAID 768 by XuanliLi-NOAA · Pull Request #966 · NOAA-EMC/GSI

XuanliLi-NOAA · 2025-12-09T21:22:41Z

Description

This PR is the continue of the closed PR 930.
This PR addresses issue #928 by adding satellite ID 768 commercial PlanetiQ YAM8 RO data to GSI v16. setupbend.f90 and read_obs.F90 have been updated to apply the commercial RO QC filters for this data, and the print statement for the number of RO obs considered and accepted has been turned off.

Resolves #928.

Type of change

Please delete options that are not relevant.

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
This change requires a documentation update

How Has This Been Tested?
GSI regression test against the develop-v16 branch (262ce16) was run on WCOSS2. All tests passed.
Test project /lfs/h2/emc/da/noscrub/xuanli.li/git/v16_768/gsi/GSI/build
Start 6: global_enkf
Start 1: global_4denvar
Start 4: hafs_4denvar_glbens
Start 5: hafs_3denvar_hybens
Start 2: rtma
Start 3: rrfs_3denvar_rdasens
1/6 Test #3: rrfs_3denvar_rdasens ............. Passed 918.86 sec
2/6 Test #2: rtma ............................. Passed 1343.99 sec
3/6 Test #5: hafs_3denvar_hybens .............. Passed 1419.25 sec
4/6 Test #4: hafs_4denvar_glbens .............. Passed 1545.86 sec
5/6 Test #6: global_enkf ...................... Passed 1786.39 sec
6/6 Test #1: global_4denvar ................... Passed 2228.87 sec
100% tests passed, 0 tests failed out of 6.

Checklist

My code follows the style guidelines of this project
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
New and existing tests pass with my changes
Any dependent changes have been merged and published

XuanliLi-NOAA · 2025-12-09T21:24:15Z

@RussTreadon-NOAA Could you please add @ADCollard and @HaixiaLiu-NOAA as the reviewers? Thank you!

RussTreadon-NOAA · 2025-12-11T14:29:39Z

Confirm that global_enkf fails when run from XuanliLi-NOAA:v16_piq768. enkf.x aborts as previously reported

 /lfs/h2/emc/ptmp/russ.treadon/pr966/tmpreg_global_enkf/global_enkf_loproc_updat
 /bfg_2024022300_fhr03_mem009
99
nid001425.cactus.wcoss2.ncep.noaa.gov: rank 18 exited with code 99
forrtl: error (78): process killed (SIGTERM)
Image              PC                Routine            Line        Source
enkf.x             0000000001AD25EB  Unknown               Unknown  Unknown
libpthread-2.31.s  00001479085D3910  Unknown               Unknown  Unknown
libmpi_intel.so.1  00001478FB38DD46  Unknown               Unknown  Unknown
libmpi_intel.so.1  00001478FB429245  Unknown               Unknown  Unknown
libmpi_intel.so.1  00001478F9C34D3E  PMPI_Barrier          Unknown  Unknown
libmpifort_intel.  00001478FFA45FCB  pmpi_barrier          Unknown  Unknown
enkf.x             00000000004C5320  gridio_mp_readgri         482  gridio_gfs.f90
enkf.x             00000000004B1B95  statevec_mp_read_         191  statevec.f90
enkf.x             0000000000413605  MAIN__                    150  enkf_main.f90

The anavinfo file in this PR does not work with GFS v16.

XuanliLi-NOAA · 2025-12-17T22:03:55Z

@RussTreadon-NOAA I have created a GSI-fix PR and updated my GSI branch. Please let me know if there's anything else I need to do.

RussTreadon-NOAA · 2025-12-17T22:08:29Z

Thank you @XuanliLi-NOAA for the updates in GSI-fix and GSI.

GSI-fix PR #58 has been approved and merged into GSI-fix develop-v16. Done at a314292.

The fix/ submodule hash in this PR needs to be updated to a314292

XuanliLi-NOAA · 2025-12-17T22:24:19Z

@RussTreadon-NOAA fix/ has been updated. Thank you!

RussTreadon-NOAA · 2025-12-18T00:42:02Z

@XuanliLi-NOAA , while we can run GSI ctests for this PR, tests may / will fail because this is a develop-v16 branch. A more important test is to run a cycle using gsi.x, enkf.x, and fix/ from XuanliLi-NOAA:v16_piq768.

Do you have a parallel in which you have been testing gps 768? If yes, you could use that parallel to run a single cycle test using XuanliLi-NOAA:v16_piq768. Ideally one would re-run the same cycle using the operational gsi.x, enkf.x, and fix/ to confirm that the only difference is that the test assimilates gps 768.

XuanliLi-NOAA · 2025-12-18T15:34:47Z

Yes, a parallel experiment, do5_allpiq (archive dir: /lfs/h2/emc/da/noscrub/[xuanli.li/archive/do5_allpiq), was completed using gfs v16.3.26, in which we assimilated all operational data plus gps 768. Compared to the operational run, gsistat for the first cycle at 2025091518 shows differences only for gpsro piq 768:

<  o-g 01     gps asm 768 0000 count         24        507        858       3083       4635       3072       2003       2343       2884       3653       5572      53022
<  o-g 01     gps asm 768 0000  bias -0.411E+00  0.167E+00 -0.133E+00 -0.543E+00 -0.198E+00 -0.147E+00 -0.143E+00 -0.182E+00 -0.995E-01  0.176E-01 -0.486E-01 -0.960E-01
<  o-g 01     gps asm 768 0000   rms  0.454E+01  0.444E+01  0.471E+01  0.456E+01  0.281E+01  0.138E+01  0.127E+01  0.119E+01  0.130E+01  0.142E+01  0.161E+01  0.212E+01
<  o-g 01     gps asm 768 0000  cpen  0.123E+00  0.207E+00  0.332E+00  0.409E+00  0.221E+00  0.278E+00  0.535E+00  0.497E+00  0.572E+00  0.671E+00  0.830E+00  0.387E+00
<  o-g 01     gps asm 768 0000 qcpen  0.123E+00  0.207E+00  0.332E+00  0.409E+00  0.221E+00  0.278E+00  0.535E+00  0.497E+00  0.572E+00  0.671E+00  0.830E+00  0.387E+00
<  o-g 01         asm all      count        163       4098       8598      33113      50709      34705      21950      25416      30758      38045      59021     617122
<  o-g 01         asm all       bias -0.168E+01 -0.560E+00  0.282E+00 -0.307E+00 -0.291E+00 -0.160E+00 -0.129E+00 -0.173E+00 -0.924E-01 -0.485E-01  0.989E-01 -0.346E-01
<  o-g 01         asm all        rms  0.535E+01  0.549E+01  0.582E+01  0.568E+01  0.345E+01  0.157E+01  0.128E+01  0.120E+01  0.125E+01  0.148E+01  0.171E+01  0.286E+01
<  o-g 01         asm all       cpen  0.257E+00  0.286E+00  0.347E+00  0.432E+00  0.308E+00  0.365E+00  0.537E+00  0.499E+00  0.526E+00  0.735E+00  0.951E+00  0.401E+00
<  o-g 01         asm all      qcpen  0.257E+00  0.286E+00  0.347E+00  0.432E+00  0.308E+00  0.365E+00  0.537E+00  0.499E+00  0.526E+00  0.735E+00  0.951E+00  0.401E+00
>  o-g 01         asm all      count        139       3591       7740      30030      46074      31633      19947      23073      27874      34392      53449     564100
>  o-g 01         asm all       bias -0.190E+01 -0.662E+00  0.328E+00 -0.282E+00 -0.300E+00 -0.161E+00 -0.128E+00 -0.172E+00 -0.916E-01 -0.555E-01  0.114E+00 -0.288E-01
>  o-g 01         asm all        rms  0.548E+01  0.562E+01  0.593E+01  0.579E+01  0.351E+01  0.159E+01  0.128E+01  0.121E+01  0.124E+01  0.149E+01  0.172E+01  0.292E+01
>  o-g 01         asm all       cpen  0.280E+00  0.297E+00  0.349E+00  0.434E+00  0.317E+00  0.374E+00  0.537E+00  0.499E+00  0.521E+00  0.742E+00  0.963E+00  0.402E+00
>  o-g 01         asm all      qcpen  0.280E+00  0.297E+00  0.349E+00  0.434E+00  0.317E+00  0.374E+00  0.537E+00  0.499E+00  0.521E+00  0.742E+00  0.963E+00  0.402E+00
<  o-g 01     gps rej 768 0000 count         38        439        526        863        986        492         75         33         25         46        178      19663
<  o-g 01     gps rej 768 0000  bias -0.670E+01 -0.694E+01 -0.694E+01 -0.607E+01 -0.186E+01 -0.644E+00  0.654E+00 -0.300E+00 -0.319E+01 -0.206E+01 -0.108E+01 -0.820E+01
<  o-g 01     gps rej 768 0000   rms  0.146E+02  0.181E+02  0.218E+02  0.214E+02  0.123E+02  0.521E+01  0.383E+01  0.436E+01  0.551E+01  0.590E+01  0.653E+01  0.530E+03
<  o-g 01     gps rej 768 0000  cpen  0.000E+00  0.000E+00  0.000E+00  0.000E+00  0.000E+00  0.000E+00  0.000E+00  0.000E+00  0.000E+00  0.000E+00  0.000E+00  0.000E+00
<  o-g 01     gps rej 768 0000 qcpen  0.000E+00  0.000E+00  0.000E+00  0.000E+00  0.000E+00  0.000E+00  0.000E+00  0.000E+00  0.000E+00  0.000E+00  0.000E+00  0.000E+00
<  o-g 01         rej all      count        516       6255       6534      10704      10153       4385        446        223        306        503       1613     166787
<  o-g 01         rej all       bias -0.607E+01 -0.538E+01 -0.417E+01 -0.595E+01 -0.278E+01 -0.449E+00  0.101E+01  0.141E+01  0.433E+00  0.124E+01  0.951E-01 -0.787E+01
<  o-g 01         rej all        rms  0.149E+02  0.163E+02  0.200E+02  0.226E+02  0.153E+02  0.631E+01  0.536E+01  0.721E+01  0.753E+01  0.837E+01  0.774E+01  0.266E+03
>  o-g 01         rej all      count        478       5816       6008       9841       9167       3893        371        190        281        457       1435     147124
>  o-g 01         rej all       bias -0.602E+01 -0.526E+01 -0.392E+01 -0.594E+01 -0.288E+01 -0.425E+00  0.108E+01  0.171E+01  0.755E+00  0.158E+01  0.240E+00 -0.783E+01
>  o-g 01         rej all        rms  0.150E+02  0.162E+02  0.198E+02  0.227E+02  0.156E+02  0.643E+01  0.562E+01  0.760E+01  0.769E+01  0.858E+01  0.787E+01  0.206E+03```

Code changes in read_obs.F90 and setupbend.f90 under /lfs/h2/emc/da/noscrub/[xuanli.li/git/do5/gfs_v16/gfs.v16.3.26/sorc/gsi-allpiq.fd/src/gsi are the same as these in XuanliLi-NOAA:v16_piq768.
I know that the current gfs has been updated to v16.3.29, should I rerun the first cycle 2025091518 with XuanliLi-NOAA:v16_piq768 and compare the gsistat?

ADCollard

Looks good and test in a cycled run is as expected

RussTreadon-NOAA · 2025-12-18T16:13:21Z

operational gfs.v16.3.29/sorc/gsi.fd -vs- XuanliLi-NOAA:v16_piq768

A diff -r of /lfs/h1/ops/prod/packages/gfs.v16.3.29/sorc/gsi.fd and /lfs/h2/emc/da/noscrub/russ.treadon/git/gsi/pr966, a local clone of XuanliLi-NOAA:v16_piq768, returns the following:

Files or directories only in one or the other

Only in pr966/: build
Only in pr966/doc: EnKF_user_guide
Only in pr966/doc: GSI_user_guide
Only in pr966/doc: README.discover
Only in pr966/doc: Release_Notes.fv3gfs_da.v15.0.0.txt
Only in pr966/doc: Release_Notes.gfsda.v16.0.0.txt
Only in /lfs/h1/ops/prod/packages/gfs.v16.3.29/sorc/gsi.fd/: exec
Only in pr966/fix: build_gsinfo
Only in pr966/fix: .gitmodules
Only in pr966/: install
Only in pr966/: regression
Only in pr966/src: GSD
Only in /lfs/h1/ops/prod/packages/gfs.v16.3.29/sorc/gsi.fd/src: ncdiag
Only in /lfs/h1/ops/prod/packages/gfs.v16.3.29/sorc/gsi.fd/src: read_atms.f90
Only in /lfs/h1/ops/prod/packages/gfs.v16.3.29/sorc/gsi.fd/src: read_gsb.f90
Only in /lfs/h1/ops/prod/packages/gfs.v16.3.29/sorc/gsi.fd/src: read_saildrone.f90
Only in pr966/: unit-tests
Only in pr966/ush: log.build
Only in pr966/ush: Testing

NCO's placement of ncdiag, read_atms.f90, read_gsb.f90, and read_saildrone in the src/ directory is odd. The other Only differences are not unusual.

Differences in files

diff '--exclude=.git' -r /lfs/h1/ops/prod/packages/gfs.v16.3.29/sorc/gsi.fd/fix/cloudy_radiance_info.txt pr966/fix/cloudy_radiance_info.txt
diff '--exclude=.git' -r /lfs/h1/ops/prod/packages/gfs.v16.3.29/sorc/gsi.fd/fix/global_convinfo.txt pr966/fix/global_convinfo.txt
diff '--exclude=.git' -r /lfs/h1/ops/prod/packages/gfs.v16.3.29/sorc/gsi.fd/fix/global_scaninfo.txt pr966/fix/global_scaninfo.txt
diff '--exclude=.git' -r /lfs/h1/ops/prod/packages/gfs.v16.3.29/sorc/gsi.fd/fix/rejectlist_global.txt pr966/fix/rejectlist_global.txt
diff '--exclude=.git' -r /lfs/h1/ops/prod/packages/gfs.v16.3.29/sorc/gsi.fd/src/gsi/qcmod.f90 pr966/src/gsi/qcmod.f90
diff '--exclude=.git' -r /lfs/h1/ops/prod/packages/gfs.v16.3.29/sorc/gsi.fd/src/gsi/read_obs.F90 pr966/src/gsi/read_obs.F90
diff '--exclude=.git' -r /lfs/h1/ops/prod/packages/gfs.v16.3.29/sorc/gsi.fd/src/gsi/setupbend.f90 pr966/src/gsi/setupbend.f90
diff '--exclude=.git' -r /lfs/h1/ops/prod/packages/gfs.v16.3.29/sorc/gsi.fd/src/gsi/setuprad.f90 pr966/src/gsi/setuprad.f90
diff '--exclude=.git' -r /lfs/h1/ops/prod/packages/gfs.v16.3.29/sorc/gsi.fd/ush/build.sh pr966/ush/build.sh

Summary

differences in global_convinfo.txt, read_obs.f90 and setupbend.f90 are from this PR. These are expected.
differences in qcmod.f90 and setuprad.f90 come from GSI PR #924. These CRTM safeguard changes are expected.
differences in cloudy_radiance_info.txt and global_scaninfo.txt are amsr2 changes from GSI-fix PR #41
differences in rejectlist_global.txt are soil DA changes from GSI-fix PR #40
differences in build.sh are from NCO adding module use /apps/ops/para/libs/modulefiles/compiler/intel/19.1.3.304/. Not sure why NCO is loading modules from apps/ops/para.

HaixiaLiu-NOAA

the changes added in read_obs.F90 and setupbend.f90 to handle the new RO data from YAM-8 with a satID=768 look good to me.

RussTreadon-NOAA · 2025-12-18T19:24:27Z

Dogwood test

Rerun the operational 20251218 00Z GDAS with the operational gsi.x (/lfs/h1/ops/prod/packages/gfs.v16.3.29/exec/gsi.x) and gsi.x from this PR. The PR gsi.x reproduced output from the operational gsi.x.

gsi.x from the PR was then rerun using fix from this PR. The gps statistics differ. When using fix from this PR, gsi.x assimilates data from gps 768. This is expected.

RussTreadon-NOAA

Approve.

RussTreadon-NOAA · 2025-12-18T19:27:03Z

Thank you @HaixiaLiu-NOAA for your review. When you have time would you please complete your review by approving the PR.

XuanliLi-NOAA · 2025-12-18T19:36:01Z

@RussTreadon-NOAA Thank you so much for taking the time to run the tests and carefully check the results. I really appreciate it.

RussTreadon-NOAA · 2025-12-18T21:55:16Z

Thank you @HaixiaLiu-NOAA for the approval.

@XuanliLi-NOAA : I'll work with the GSI Handling Review team to get this PR merged.

XuanliLi-NOAA · 2025-12-18T21:56:39Z

@RussTreadon-NOAA Thank you so much!

add PlanetiQ YAM-8 data SAID 768

07cc2a2

RussTreadon-NOAA requested review from ADCollard and HaixiaLiu-NOAA December 9, 2025 21:27

RussTreadon-NOAA assigned XuanliLi-NOAA Dec 9, 2025

RussTreadon-NOAA self-requested a review December 11, 2025 13:36

RussTreadon-NOAA mentioned this pull request Dec 17, 2025

updates for 2m obs, plus reanalysis ob input files (already in main) NOAA-PSL/build_gsinfo-fix#16

Merged

update submodule to latest version

f030b70

Update gsi-fix

5176a0b

ADCollard approved these changes Dec 18, 2025

View reviewed changes

HaixiaLiu-NOAA reviewed Dec 18, 2025

View reviewed changes

RussTreadon-NOAA approved these changes Dec 18, 2025

View reviewed changes

Comment thread src/gsi/setupbend.f90

Comment thread src/gsi/setupbend.f90

Comment thread fix

HaixiaLiu-NOAA approved these changes Dec 18, 2025

View reviewed changes

RussTreadon-NOAA merged commit 658a1e5 into NOAA-EMC:develop-v16 Dec 18, 2025

RussTreadon-NOAA mentioned this pull request Dec 18, 2025

Include commercial RO data from PlaneitQ new YAM-8 satellite #928

Closed

This was referenced May 8, 2026

Bring bug fixes into develop-v16 for minor upgrade #1006

Open

GSI changes for GFSv16.3.31 #1008

Merged

Conversation

XuanliLi-NOAA commented Dec 9, 2025

Uh oh!

XuanliLi-NOAA commented Dec 9, 2025

Uh oh!

RussTreadon-NOAA commented Dec 11, 2025

Uh oh!

XuanliLi-NOAA commented Dec 17, 2025

Uh oh!

RussTreadon-NOAA commented Dec 17, 2025

Uh oh!

XuanliLi-NOAA commented Dec 17, 2025

Uh oh!

RussTreadon-NOAA commented Dec 18, 2025

Uh oh!

XuanliLi-NOAA commented Dec 18, 2025 • edited by RussTreadon-NOAA Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ADCollard left a comment

Choose a reason for hiding this comment

Uh oh!

RussTreadon-NOAA commented Dec 18, 2025

Uh oh!

HaixiaLiu-NOAA left a comment

Choose a reason for hiding this comment

Uh oh!

RussTreadon-NOAA commented Dec 18, 2025

Uh oh!

RussTreadon-NOAA left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

RussTreadon-NOAA commented Dec 18, 2025

Uh oh!

XuanliLi-NOAA commented Dec 18, 2025

Uh oh!

RussTreadon-NOAA commented Dec 18, 2025

Uh oh!

XuanliLi-NOAA commented Dec 18, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

XuanliLi-NOAA commented Dec 18, 2025 •

edited by RussTreadon-NOAA

Loading