AHI Cloud Detection based on IR Cloud Mask by xdmjolly · Pull Request #1139 · wrf-model/WRF

xdmjolly · 2020-03-20T01:39:24Z

TYPE: new feature

KEYWORDS: WRFDA, AHI, Cloud detection

SOURCE: Deqin Li (Liaoning Meteorological Bureau of CMA/NCAR), Dongmei Xu (NUIST/NCAR), Jonathan Guerrette (NCAR).

DESCRIPTION OF CHANGES:
A cloud detection method is added for AHI radiance data assimilation based on an infrared-only cloud mask (CM) algorithm.

LIST OF MODIFIED FILES:
A: var/da/da_physics/da_trop_wmo.inc
A: var/da/da_radiance/mod_clddet_geoir.f90
M: Registry/registry.var
M: var/build/da.make
M: var/build/depend.txt
M: var/da/da_define_structures/da_define_structures.f90
M: var/da/da_physics/da_physics.f90
M: var/da/da_radiance/da_allocate_rad_iv.inc
M: var/da/da_radiance/da_crtm.f90
M: var/da/da_radiance/da_deallocate_radiance.inc
M: var/da/da_radiance/da_get_innov_vector_crtm.inc
M: var/da/da_radiance/da_initialize_rad_iv.inc
M: var/da/da_radiance/da_qc_ahi.inc
M: var/da/da_radiance/da_radiance.f90
M: var/da/da_radiance/da_radiance1.f90
M: var/da/da_radiance/da_read_obs_netcdf4ahi_jaxa.inc
M: var/da/da_radiance/da_setup_radiance_structures.inc

TESTS CONDUCTED:
WRFDA regression tests ran successfully with gnu/ifort+mpt on Cheyenne.

RELEASE NOTE: New capability for an IR-only cloud detection scheme for AHI radiance data assimilation.
Wu et al., 2020, Improving forecasts of a record-breaking rainstorm in Guangzhou by assimilating every 10-min AHI radiances with WRF 4DVAR. Atmospheric Research, 239, 104912.

Zhuge, X., Zou, X., 2016. Test of a modified infrared-only ABI cloud mask algorithm for
AHI radiance observations. J. Appl. Meteorol. Climatol. 55 (11), 2529–2546.

davegill · 2020-03-20T03:09:18Z

@xdmjolly
Please add the text of the jenkins email in one of these lower comments

liujake · 2020-03-25T23:41:24Z

Registry/registry.var

 rconfig   integer   varbc_nbgerr            namelist,wrfvar14  1  5000     - "varbc_nbgerr"            ""  ""
 rconfig   integer   varbc_nobsmin           namelist,wrfvar14  1  10       - "varbc_nobsmin"           ""  ""
 rconfig   integer   use_clddet              namelist,wrfvar14  1  2        - "use_clddet"        "0: off, 1: mmr, 2: pf, 3: ecmwf"  ""
+rconfig   logical   use_clddet_zz           namelist,wrfvar14  1  .false.  - "use_clddet_zz"           ""  ""


what means '_zz'? Zhuge&Zou (2016)?, If so, please add some description inside " ".

liujake · 2020-03-25T23:42:37Z

var/build/da.make

   gsi_kinds.o \
   gsi_constants.o \
   gsi_thinning.o \
+   mod_qc.o \


The name of mod_qc is too generic for people to know what this is for. Can you use some more meaningful file name?

liujake · 2020-03-25T23:46:00Z

var/da/da_define_structures/da_define_structures.f90

+   type cld_qc_type
+     real  :: RTCT, RFMFT, TEMPIR, terr_hgt
+     real  :: tb_stddev_10, tb_stddev_13,tb_stddev_14
+     real  :: CIRH2O
+     !real, allocatable :: CIRH2O(:,:,:)
+   end type cld_qc_type   


if this cloud detection is specific to AHI (and ABI?), better to make the type name more specific, e.g., something like 'clddet_geoir'.

liujake · 2020-03-25T23:47:50Z

var/da/da_define_structures/da_define_structures.f90

      real,    pointer     :: satazi(:) 
      real,    pointer     :: solzen(:) 
      real,    pointer     :: solazi(:) 
+	  real,    pointer     :: tropt(:)


please add a comment to tell this what this variable is? 'tropopause temperature'?

I'm encountering difficulties in assimilating sunflower data. I've adopted the cloud detection method proposed by Zhuge and Zou, but I'm unable to obtain the "tropt" (tropopause temperature) data, which is preventing me from completing the drawing of the cloud detection map. Does anyone know how this data is obtained?

liujake · 2020-03-25T23:55:08Z

var/da/da_radiance/da_qc_ahi.inc

   ! Purpose: perform quality control for ahi  data.
   !  Method: Assume cloud flag coming from GEOCAT processing
+   ! To be developed: built in cloud_detection method 
+   ! HISTORY: 2020/03/01 - Add clear sky cloud detection procedures   Dongmei Xu, NUIST/CAS, NCAR/NESL/MMM/DAS   


no NESL any more! no DAS any more. You can just use NCAR/MMM

liujake · 2020-03-26T00:00:46Z

var/da/da_radiance/da_qc_ahi.inc

-
-         ! 1. Cloud detection scheme MMR in Auligné (2014).or. PF in Xu et al., (2016)
-         !---------------------------------------------
-         if ((use_clddet==1 .or. use_clddet==2) .and. (.not.use_satcv(2))) then
-            iv%instid(i)%cloud_flag(:,n) = qc_good
-


why you delete the existing cloud detection schemes? Can we keep them?

liujake · 2020-03-26T00:04:59Z

var/da/da_radiance/da_read_obs_netcdf4ahi_jaxa.inc

+   !  Purpose: read in JAXA AHI Level-1 and Level-1 (previous) data in NETCDF4 format
   !           and form innovation structure


Unclear what is the difference between 'Level-1' and 'Level-1 (previous)'. Please make it more understandable.

liujake · 2020-03-26T00:06:01Z

var/da/da_radiance/da_read_obs_netcdf4ahi_jaxa.inc

+   !  HISTORY: 2020/03/01 - Add clear sky cloud detection procedures   Dongmei Xu, NUIST/CAS, NCAR/NESL/MMM/DAS  
   !  To be devoloped: 1.time information; 2.dimension sequence


Same comment about affiliation here.

liujake · 2020-03-26T00:11:42Z

var/da/da_radiance/mod_qc.f90

+  module mod_qc
+!   use netcdf


I would use a more relevant name like mod_clddet_ahi or sth more general mod_clddet_geoir if it is applicable to both AHI and ABI.

Plantain · 2021-09-13T01:50:14Z

This commit appears to depend on a static file "static_ahi.nc" in da_read_obs_netcdf4ahi_jaxa.inc
@xdmjolly @liujake where can this file be found?

It also looks like this commit breaks using JAXA NetCDF4's for usecases that don't use use_clddet_zz, see cases of this popping up in the forums like https://forum.mmm.ucar.edu/phpBB3/viewtopic.php?f=18&t=9512&p=20935#p20935
https://forum.mmm.ucar.edu/phpBB3/viewtopic.php?t=10206

liujake · 2021-09-20T19:25:58Z

@Plantain Somehow this cloud detection scheme was omitted in WRFDA 4.2 release note. It is now added in https://www2.mmm.ucar.edu/wrf/users/wrfda/updates-4.2.html. The download link for static_ahi.bc is provided.

Plantain · 2021-09-20T21:25:15Z

@liujake Thank you!

I would again draw your attention though to this change breaking any use of JAXA Himawari8 NetCDF's with anything other than clddet_zz though, due to the changes in https://github.com/wrf-model/WRF/pull/1139/files#diff-4a0f1b1feda554b73ca5fa2f34c70168af0e721800b791a6d66f4fe0dffd274bR237 and da_read_obs_netcdf4ahi_jaxa.inc.

Currently if clddet_zz = 0, WRFDA will crash.

I don't think that was an intended part of this change!

TYPE: new feature KEYWORDS: ABI, cloud detection, all-sky obs error model SOURCE: JJ Guerrette (NCAR/MMM, now at tomorrow.io), Deqin Li (Liaoning Meteorological Bureau of CMA), Jake Liu (NCAR/MMM) DESCRIPTION OF CHANGES: This PR addes the assimilation of GOES-16/17 ABI's 3 water vapor channels' radiance data. This includes reading of ABI's full-disk, CONUS, and meso1&2 data files, superobbing and thinning of ABI data, IR-based cloud detection scheme as part of quality control, and all-sky obs error model. Cloud detection scheme should be the same as for AHI in principle, but the actual code implementation is not the same. No attempt made to make ABI's cloud detection code consistent with AHI's when bringing the code originally developed back in 2019-2020 into the latest develop branch. Some technical information is provided below for the use of this new capability. 1. Read ABI files: Raw netcdf ABI data files (one file for one channel) need to be listed in 4 list files: 'file_list_GOES-16-ABI_C' for CONUS scan files 'file_list_GOES-16-ABI_F' for full-disk scan files (e.g., OR_ABI-L1b-RadF-M6C08_G16_s20191211200263_e20191211209571_c20191211210021.nc) 'file_list_GOES-16-ABI_M1' for meso1 scan files 'file_list_GOES-16-ABI_M2' for meso2 scan files ABI data reading code will automatically determine which file(s) to read in matching ABI file's time and analysis time. 2. Cloud detection scheme needs to read in a terrain file OR_ABI-TERR_G16.nc or OR_ABI-TERR_G17.nc for GOES-16 ABI or GOES-17 ABI. 3. Related namelist settings: ``` &wrfvar4 use_goesabiobs = true, ! read goes-16 and goes-17 ABI data / ``` ``` &wrfvar14 rtminit_nsensor= 1, rtminit_platform= 4, ! goes rtminit_satid= 16, ! goes-16 rtminit_sensor= 44, ! abi thinning= true, thinning_mesh= 30.0, qc_rad=true, write_iv_rad_ascii=true, write_oa_rad_ascii=true, rtm_option= 2, crtm_cloud= false, only_sea_rad=false, use_varbc=true, varbc_nobsmin=500, crtm_irland_coef= "IGBP.IRland.EmisCoeff.bin", use_clddet_zz=true, ! IR-based cloud detection abi_superob_halfwidth=3, ! this will do supperobbing with 7x7 pixels / ``` See also AHI DA related PRs: #1139 #1173 #1774 LIST OF MODIFIED FILES: 41 M Registry/registry.var M var/build/depend.txt M var/da/da_define_structures/da_define_structures.f90 M var/da/da_monitor/da_rad_diags.f90 M var/da/da_radiance/da_allocate_rad_iv.inc M var/da/da_radiance/da_deallocate_radiance.inc M var/da/da_radiance/da_get_innov_vector_crtm.inc M var/da/da_radiance/da_get_innov_vector_rttov.inc A var/da/da_radiance/da_get_sat_angles.inc A var/da/da_radiance/da_get_sat_angles_1d.inc A var/da/da_radiance/da_get_solar_angles.inc A var/da/da_radiance/da_get_solar_angles_1d.inc M var/da/da_radiance/da_initialize_rad_iv.inc A var/da/da_radiance/da_qc_goesabi.inc M var/da/da_radiance/da_qc_rad.inc M var/da/da_radiance/da_radiance.f90 M var/da/da_radiance/da_radiance1.f90 M var/da/da_radiance/da_radiance_init.inc A var/da/da_radiance/da_read_obs_ncgoesabi.inc M var/da/da_radiance/da_rttov.f90 M var/da/da_radiance/da_setup_radiance_structures.inc M var/da/da_radiance/da_write_iv_rad_ascii.inc M var/da/da_radiance/da_write_oa_rad_ascii.inc M var/da/da_radiance/module_radiance.f90 M var/da/da_setup_structures/da_setup_obs_structures.inc M var/da/da_setup_structures/da_setup_structures.f90 A var/da/da_tools/da_llxy_1d.inc A var/da/da_tools/da_llxy_default_1d.inc A var/da/da_tools/da_llxy_global_1d.inc A var/da/da_tools/da_llxy_kma_global_1d.inc A var/da/da_tools/da_llxy_latlon_1d.inc A var/da/da_tools/da_llxy_lc_1d.inc A var/da/da_tools/da_llxy_merc_1d.inc A var/da/da_tools/da_llxy_ps_1d.inc A var/da/da_tools/da_llxy_rotated_latlon_1d.inc A var/da/da_tools/da_llxy_wrf_1d.inc A var/da/da_tools/da_togrid_1d.inc M var/da/da_tools/da_tools.f90 M var/run/VARBC.in A var/run/radiance_info/goes-16-abi.info A var/run/radiance_info/goes-17-abi.info TESTS CONDUCTED: 1. WRFDA regression test passed on Derecho. 2. Clear-sky ABI DA is tested with a full-disk data file. 3. the Jenkins tests all passing. RELEASE NOTE: Add the capability for assimilating GOES-ABI radiance data

TYPE: new feature KEYWORDS: WRFDA, AHI, Cloud detection SOURCE: Deqin Li (Liaoning Meteorological Bureau of CMA/NCAR), Dongmei Xu (NUIST/NCAR), Jonathan Guerrette (NCAR). DESCRIPTION OF CHANGES: A cloud detection method is added for AHI radiance data assimilation based on an infrared-only cloud mask (CM) algorithm. LIST OF MODIFIED FILES: A: var/da/da_physics/da_trop_wmo.inc A: var/da/da_radiance/mod_clddet_geoir.f90 M: Registry/registry.var M: var/build/da.make M: var/build/depend.txt M: var/da/da_define_structures/da_define_structures.f90 M: var/da/da_physics/da_physics.f90 M: var/da/da_radiance/da_allocate_rad_iv.inc M: var/da/da_radiance/da_crtm.f90 M: var/da/da_radiance/da_deallocate_radiance.inc M: var/da/da_radiance/da_get_innov_vector_crtm.inc M: var/da/da_radiance/da_initialize_rad_iv.inc M: var/da/da_radiance/da_qc_ahi.inc M: var/da/da_radiance/da_radiance.f90 M: var/da/da_radiance/da_radiance1.f90 M: var/da/da_radiance/da_read_obs_netcdf4ahi_jaxa.inc M: var/da/da_radiance/da_setup_radiance_structures.inc TESTS CONDUCTED: WRFDA regression tests ran successfully with gnu/ifort+mpt on Cheyenne. RELEASE NOTE: New capability for an IR-only cloud detection scheme for AHI radiance data assimilation. Wu et al., 2020, Improving forecasts of a record-breaking rainstorm in Guangzhou by assimilating every 10-min AHI radiances with WRF 4DVAR. Atmospheric Research, 239, 104912. Zhuge, X., Zou, X., 2016. Test of a modified infrared-only ABI cloud mask algorithm for AHI radiance observations. J. Appl. Meteorol. Climatol. 55 (11), 2529–2546.

Dongmei Xu and others added 2 commits March 6, 2020 13:27

qcahi

6fbee46

Add zz-cloud detection

3623c15

xdmjolly requested a review from a team as a code owner March 20, 2020 01:39

ahi clddet

e512151

davegill changed the title ~~Ahi clddet~~ AHI Cloud Detection based on IR Cloud Mask Mar 20, 2020

davegill added WRFDA New Feature Develop Branch labels Mar 20, 2020

liujake requested changes Mar 26, 2020

View reviewed changes

Dongmei Xu added 2 commits March 25, 2020 21:14

ehancement

7ddd557

ehancement

161ad6d

liujake approved these changes Mar 27, 2020

View reviewed changes

liujake merged commit d96531b into wrf-model:develop Mar 27, 2020

This was referenced Jan 16, 2024

Add the capability for assimilating GOES-ABI radiance data liujake/WRF#1

Open

Add the capability for assimilating GOES-ABI radiance data #1983

Merged

		! Purpose: read in JAXA AHI Level-1 and Level-1 (previous) data in NETCDF4 format
		! and form innovation structure

		! HISTORY: 2020/03/01 - Add clear sky cloud detection procedures Dongmei Xu, NUIST/CAS, NCAR/NESL/MMM/DAS
		! To be devoloped: 1.time information; 2.dimension sequence

Conversation

xdmjolly commented Mar 20, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

davegill commented Mar 20, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Plantain commented Sep 13, 2021

Uh oh!

liujake commented Sep 20, 2021

Uh oh!

Plantain commented Sep 20, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

xdmjolly commented Mar 20, 2020 •

edited

Loading