Add capability to ingest and assimilate IASI-NG

[wx20jjung]

Description

These are various changes needed to connect the IASI-NG data files to the GSI and ENKF routines. They include connecting the RARS/EARS and DBNet data when available.

Data links are contained in the exglobal_atmos_analysis.sh. mtias2 is the operational data file. Esias2 and ias2db lines are for the RARS/EARS and DBNet data files.
IASINGBF=${IASINGBF:-${COM_OBS}/${OPREFIX}mtias2.tm00.bufr_d${OSUFFIX}}
ESIASING=${ESIASING:-${COM_OBS}/${OPREFIX}esias2.tm00.bufr_d${OSUFFIX}}
IASINGDB=${IASINGDB:-${COM_OBS}/${OPREFIX}ias2db.tm00.bufr_d${OSUFFIX}}

Eventually the cloud and aerosol detection software (CADS) will be used for cloud detection. There is an auxiliary file to modify several default parameters, if necessary. This file is also linked in the exglobal_atmos_analysis.sh file. This IASING_CLDDET.NL file is not required at run time and is ignored if the file does not exist.
${NLN} ${FIXgfs}/gsi/IASING_CLDDET.NL IASING_CLDDET.NL

There is a flag in the config.anal file to turn on/off the CADS. This flag is turned on/off with a namelist variable in OBSQC
iasing_cads=${IASING_CADS}

If this iasing_cads flag (above) is incorporated before the changes in the GSI (added to GSI on 11/25) are available, this will break the global workflow.

A line is added to obs_input to define the data thinning and the surface calculations. These values are consistent with the current IASI and CrIS instruments.
iasingbufr iasi-ng metop-sg-a1 iasi-ng_metop-sg-a1 0.0 1 0

A line is added to the exgdas_enkf_update.sh to use these data within the ensembles similar to exglobal_atmos_analysis.sh.
sattypes_rad(80)= 'iasi-ng_metop-sg-a1', dsis(80)= 'iasi-ng_metop-sg-a1',

Iasi-ng_metop-sg-a1 was appended to the diagtype[3} variable in exglobal_diag.sh to collect O-B and O-A statistics for iasi-ng.

There are fix files that will be required to eventually use IASI-NG. They are global_satinfo.txt, global_scaninfo.txt and IASING_CLDDET.NL. The global_satinfo.txt will contain the assumed 500 channel subset expected to be used. The global_scaninfo.txt contains the information if fields-of-view along the edges of the scan line are to be removed. IASING_CLDDET.NL contains run-time namelist changes for the cloud and aerosol detection software (CADS). Global_satinfo.txt should be installed after the IASI-NG and MetImage coefficient files exist in the CRTM fix directory. The GSI has internal defaults for scaninfo.txt and IASING_CLDDET.NL and do not need to exist.

Resolves #3081
Depends on NOAA-EMC/GSI#804

Type of change

• Bug fix (fixes something broken)
• New feature (adds functionality)
• Maintenance (code refactor, clean-up, new CI test, etc.)

Change characteristics

• Is this a breaking change (a change in existing functionality)? NO
• Does this change require a documentation update? NO
• Does this change require an update to any of the following submodules? YES
  • GSI Add capability to read IASI-NG GSI#804

How has this been tested?

I have run cycled tests with these changes using IASI-NG proxy data on Hera.
I have also conducted limited unit tests with my changes within the global-workflow:
C96C48_hybatmDA
C96_atm3DVar
Both pass when the OBSPROC namelist variable is removed or the latest develop branch of the GSI is used.

Checklist

• Any dependent changes have been merged and published
• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have documented my code, including function, input, and output descriptions
• My changes generate no new warnings
• New and existing tests pass with my changes
• This change is covered by an existing CI test or a new one has been added
• Any new scripts have been added to the .github/CODEOWNERS file with owners
• I have made corresponding changes to the system documentation if necessary

[WalterKolczynski-NOAA]

@wx20jjung Is there any reason this is still in draft, or is it ready for review? Shellcheck failures should be corrected.

[wx20jjung]

Walter, This pull request requires the latest GSI 'develop'. There are namelist variables that are set in the scripts but not yet in the GSI the global-workflow is using. Jim

…

On Tue, Dec 10, 2024 at 2:48 PM Walter Kolczynski - NOAA < ***@***.***> wrote: @wx20jjung <https://github.com/wx20jjung> Is there any reason this is still in draft, or is it ready for review? Shellcheck failures should be corrected. — Reply to this email directly, view it on GitHub <#3126 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AMPASAYUNMZGIYTKFCYFOYD2E5ARPAVCNFSM6AAAAABSRGV2VOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMZSG4ZDANZUGI> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[KateFriedman-NOAA]

@wx20jjung Checking in on the status of this. Let us know, thanks!

[wx20jjung]

Kate, This pull request requires a new version of the CRTM. Specifically it needs spectral and tau coefficient files for IASI-NG and MetImage. These files exist in a release but are not in your current spack-stack. Dave Huber mentioned a new CRTM version will not be updated until sometime in February. Thanks, Jim

…

On Mon, Jan 6, 2025 at 10:05 AM Kate Friedman ***@***.***> wrote: @wx20jjung <https://github.com/wx20jjung> Checking in on the status of this. Let us know, thanks! — Reply to this email directly, view it on GitHub <#3126 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AMPASA2Z2FLSTSVDCWLDWRL2JKLUZAVCNFSM6AAAAABSRGV2VOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNZTGMYDAOJXHE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[KateFriedman-NOAA]

Noted, thanks @wx20jjung !

[DavidHuber-NOAA]

@wx20jjung Would it be alright with you if we close this PR for now and reopen when spack-stack 1.9.1 is incorporated and the upstream GSI is ready to go?

[wx20jjung]

Dave, Yes, close this ticket. Jim

…

On Tue, Mar 11, 2025 at 2:07 PM David Huber ***@***.***> wrote: @wx20jjung <https://github.com/wx20jjung> Would it be alright with you if we close this PR for now and reopen when spack-stack 1.9.1 is incorporated and the upstream GSI is ready to go? — Reply to this email directly, view it on GitHub <#3126 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AMPASA6X7GEUSXRIZSH2LUT2T4Q67AVCNFSM6AAAAABSRGV2VOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMJVGI3DMNBZGQ> . You are receiving this because you were mentioned.Message ID: ***@***.***> [image: DavidHuber-NOAA]*DavidHuber-NOAA* left a comment (NOAA-EMC/global-workflow#3126) <#3126 (comment)> @wx20jjung <https://github.com/wx20jjung> Would it be alright with you if we close this PR for now and reopen when spack-stack 1.9.1 is incorporated and the upstream GSI is ready to go? — Reply to this email directly, view it on GitHub <#3126 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AMPASA6X7GEUSXRIZSH2LUT2T4Q67AVCNFSM6AAAAABSRGV2VOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMJVGI3DMNBZGQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>