Skip to content

A new version of the FAST (33 bins) Spectral-Bin Microphysics (FSBM)#1085

Closed
smileMchen wants to merge 33 commits intowrf-model:developfrom
smileMchen:BEFORE_SPACE_REMOVAL
Closed

A new version of the FAST (33 bins) Spectral-Bin Microphysics (FSBM)#1085
smileMchen wants to merge 33 commits intowrf-model:developfrom
smileMchen:BEFORE_SPACE_REMOVAL

Conversation

@smileMchen
Copy link
Collaborator

@smileMchen smileMchen commented Feb 10, 2020

TYPE: enhancement / new features

KEYWORDS: cloud microphysics, polarimetric-forward-operator

SOURCE: Jacob Shpund, Alexander Khain, and Barry Lynn (The Hebrew University of Jerusalem)

DESCRIPTION OF CHANGES: A new updated FSBM cloud microphysical scheme.
There are updates across all the FSBM microphysical scheme including: switch to use either graupel or hail, condensation/evaporation, nucleation (cloud base nucleation, 3 log-normal user-defined aerosol distribution), adaptive cond./evap. time-step, updated collision-coalescence, spontaneous rain breakup, spontaneous snow breakup.
A forward polarimetric operator is coupled to the FSBM scheme, where the user can see the total reflectivity field, as well as the per hydrometeor total reflectivity (rain, snow, graupel/hail). Please see mandatory input tables info in the RELEASE NOTE.

Modified PR from the original #848 A new version of the FAST (33 bins) Spectral-Bin
Microphysics (FSBM). Mostly inadvertent white space removed.

LIST OF MODIFIED FILES:
M Makefile
M Registry/registry.em_shared_collection
A Registry/registry.polrad
M Registry/registry.sbm
M arch/postamble
M dyn_em/solve_em.F
M phys/Makefile
M phys/module_diag_misc.F
M phys/module_diagnostics_driver.F
M phys/module_microphysics_driver.F
A phys/module_mp_SBM_polar_radar.F
M phys/module_mp_fast_sbm.F
M phys/module_physics_init.F

TESTS CONDUCTED: The WTF has not been used yet for the updated FSBM code.
However, the code has been compiled and used successfully for 10h of deep convection of the 20th May 2011 MC3E campaign test case (a manuscript consist of scheme details has been submitted for publication):

  • Compiled using Intel compiler 2019 (ifort version 19.0.3.199)
  • Configured using options 15 INTEL (ifort/icc, dmpar) and nest option 1.
  • BC/IC used the NCEP FNL (Final) Operational Global Analysis data are on 1-degree by 1-degree grids
  • Domain setup and time step (5s per 1.3 km):

&domains
time_step = 15,
time_step_fract_num = 0,
time_step_fract_den = 1,
max_dom = 2,
dx = 4000, 1333.33,
dy = 4000, 1333.33,
grid_id = 1, 2,
parent_id = 0, 1,
parent_grid_ratio = 1, 3,
parent_time_step_ratio = 1, 3,
...

  • In order to use the bin-wise polarimetric forward operator, a new flag must exist in the <namelist.input> file:

&physics
sbm_diagnostics = 1, 1,

  • The FAST SBM code is not currently configure to work with default 8-byte reals.

RELEASE NOTE:
Fast (33bins) Spectral-bin Microphysics (FSBM): in order to run the new FSBM scheme, users needs to download an external directory named "SBM_input_33" consist of mandatory input tables and place it in the 'run' directory.
In case the coupled polarimetric forward operator is to be used (e.g., 'sbm_diagnostics = 1'), a second directory consist of scattering amplitudes named "SBM_scatter_amplit.tgz" needs to be placed in the 'run' directory.
Both directories are compressed and can be downloaded at the following link:
https://drive.google.com/drive/folders/1qxYyQwKI1wKQYasDUkQvVgHs11prLiqA?usp=sharing
The FAST SBM code is not currently configure to work with default 8-byte reals.

@smileMchen smileMchen requested review from a team as code owners February 10, 2020 22:55
@smileMchen smileMchen changed the title Before space removal AFTER space removal Feb 10, 2020
@davegill davegill changed the title AFTER space removal AFTER space removal - FSBM candidate Feb 10, 2020
@davegill davegill changed the title AFTER space removal - FSBM candidate AFTER space removal - FSBM candidate alternate Feb 10, 2020
@JS-WRF
Copy link

JS-WRF commented Feb 12, 2020

@davegill @smileMchen @dudhia
Dave, Ming and Jimy -- thanks a lot for pushing this forward in a short time.
After reviewing the changes, I have several questions:

  1. In the you are linking the SBM_input_33 which is fine, but also the scattering tables which we agreed will be at an external link. Did you decided to include the this tables directory?
    This table is for S-band radar (11cm wavelength; standard for WSR-88D in the US). However other wavelength could be used as well. This could be addressed as well in the next bug fix, where I'll provide other wavelength tables (maybe put these in external links).

  2. What's the easiest way to clone this new PR version (A new version of the FAST (33 bins) Spectral-Bin Microphysics (FSBM) #1085 ), so I can review it (mainly the <module_mp_fast_sbm.F> file which has multiple differences). I would like also to run the MC3E test case as Dave suggested.

  3. How different will the "release notes" be for users of this version until the next bug fix (which hopefully will enable us to use the polarimetric operator)?

@dudhia
Copy link
Collaborator

dudhia commented Feb 12, 2020 via email

@JS-WRF
Copy link

JS-WRF commented Feb 13, 2020

@dudhia @davegill @smileMchen

Jimy -- got it, thanks.
Dave / Ming -- while configuring for idealize/real cases (option 13/0, 15/1), the compilation aborted for the radiation driver module:

module_radiation_driver.f90(2001): error #5082: Syntax error, found '.OR.' when expecting one of: ( <IDENTIFIER> <CHAR_CON_KIND_PARAM> <CHAR_NAM_KIND_PARAM> <CHARACTER_CONSTANT> <INTEGER_CONSTANT> ...
                              .OR. sw_physics.eq.RRTMK_SWSCHEME &
------------------------------^
compilation aborted for module_radiation_driver.f90 (code 1)

The configuration includes the default settings:
-DBUILD_BUILD_POLAR_HUCM=0

Compiler info:

ifort (IFORT) 19.0.3.199 20190206
Copyright (C) 1985-2019 Intel Corporation.  All rights reserved. 

@dudhia
Copy link
Collaborator

dudhia commented Feb 13, 2020 via email

@dudhia
Copy link
Collaborator

dudhia commented Feb 13, 2020 via email

@smileMchen
Copy link
Collaborator Author

@JS-WRF @dudhia @davegill
There is an extra .OR. in the code phys/module_radiation_driver.F, which caused the error in compiling. I will fix this.

@smileMchen
Copy link
Collaborator Author

The code can be compiled successfully now.

@JS-WRF
Copy link

JS-WRF commented Feb 13, 2020

@smileMchen @davegill @dudhia

Ming, thank you.
The regular compilation (for idealized case, 13/0; in debug mode "./configure -D") indeed compiles.
However, the regression test ("./configure -r8") doesn't pass, as the FSBM produce multiple type conflicts. I'm not sure what the regression check does, but the standard build compilation should have catch this as well, but didn't as mentioned.
I'm going on board a flight to SF right now, and will hardly be available for the next 24 hours or so.
Can you please double check what's going on?

Thanks,
Kobby

@dudhia
Copy link
Collaborator

dudhia commented Feb 13, 2020 via email

@smileMchen
Copy link
Collaborator Author

@kobby,
I just compiled this code with -r8. Below is what I did:

./clean -a
./configure -r8
./compile em_real &> r8.log&

It is successfully compiled. ( I compile with gnu/9.1.0)

@dudhia
Copy link
Collaborator

dudhia commented Feb 13, 2020 via email

@JS-WRF
Copy link

JS-WRF commented Feb 15, 2020

@smileMchen @davegill @dudhia

Compiling with Intel, the ideal as well as the real regression test fails for the FSBM module.
./configure -r8 (options 13/0)
./configure -r8 (option 15/1)
The corresponding files are attached.

Please advise.

cmp_ideal_R.log
cmp_real_R.log

@JS-WRF
Copy link

JS-WRF commented Feb 19, 2020

@smileMchen @davegill @dudhia

Folks,
Just making sure you saw my message.

@davegill
Copy link
Contributor

@smileMchen
Ming,
Would you please try to build this repo on cheyenne with the default intel compiler 18.0.5.

!
INTEGER, INTENT(IN ) :: ids,ide, jds,jde, kds,kde
INTEGER, INTENT(IN ) :: ims,ime, jms,jme, kms,kme,num_scalar
INTEGER, INTENT(IN ) :: ims,ime, jms,jme, kms,kme,num_scalar, num_sbmradar
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does num_sbmradar come into this driver? This may need to be declared 'optional'.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

num_sbmradar cannot be declared as optional because it is used as the dimension of the array sbmradar


REAL, DIMENSION( ims:ime , kms:kme , jms:jme ),INTENT(INOUT), OPTIONAL :: th_old,qv_old
REAL,DIMENSION(ims:ime,kms:kme,jms:jme,num_scalar),INTENT(INOUT), OPTIONAL :: scalar
REAL, DIMENSION(ims:ime,kms:kme,jms:jme,num_sbmradar),INTENT(INOUT) :: sbmradar
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see how sbmradar comes into this module either. And this also needs to be declared 'optional'.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@weiwangncar

Indeed, the SBMRADAR needs to be defined as "optional".
It's dimension and definition are derived from the Registry file (at least to my knowledge):
Registry/registry.polrad

smileMchen and others added 3 commits February 19, 2020 14:23
The use of 4-byte and 8-byte reals in the FAST SBM scheme is
intermixed with `double precision`. The code works correctly for
default 4-byte reals. However, the code will not compile when
setting 8-byte reals as the default.

The solution is to ifdef out the entire FAST SBM code when
building with -r8.

Changes to be committed:
modified:   arch/postamble
modified:   configure
modified:   phys/module_microphysics_driver.F
modified:   phys/module_mp_SBM_polar_radar.F
modified:   phys/module_mp_fast_sbm.F
modified:   phys/module_physics_init.F
modified:   share/module_check_a_mundo.F

The code builds successfully on Linux and Mac (both with GNU 7).
Both OS options were tested with and without -r8. All tests were
conducted with mp_physics = 30 (FAST SBM).
@davegill
Copy link
Contributor

davegill commented Feb 20, 2020

@smileMchen @weiwangncar
Folks,
We need to resolve the conflict.
It looks like the mods in the old diag_misc need to be moved to diag_nwp.
I do not seem to be able to get a jenkins test. On the classroom, my mods worked OK with -r8.

Upon further review, I caused the "no test" condition from Jenkins when I disabled some user privileges.

Changes to be committed:
modified:   module_microphysics_driver.F
modified:   module_mp_fast_sbm.F
Changes to be committed:
modified:   module_mp_fast_sbm.F
@davegill
Copy link
Contributor

@smileMchen @weiwangncar @dudhia
Folks,
The FSBM is internally OK:

    Test Type              | Expected  | Received |  Failed
    = = = = = = = = = = = = = = = = = = = = = = = =  = = = =
    Number of Tests        : 10           10
    Number of Builds       : 23           23
    Number of Simulations  : 65           65        0
    Number of Comparisons  : 39           39        0

    Failed Simulations are: 
    None
    Which comparisons are not bit-for-bit: 
    None

However, we need to move some diagnostic code from diag_misc to diag_nwp to fix a conflict. Other than fixing the conflict, I think that we are good to go.

@davegill
Copy link
Contributor

Now using #1097 "A new version of the FAST (33 bins) Spectral-Bin Microphysics (FSBM)"

Fixed the conflicts without trashing someone else's repo.

@davegill davegill closed this Feb 20, 2020
davegill added a commit that referenced this pull request Feb 27, 2020
…1097)

TYPE: enhancement / new features

KEYWORDS: cloud microphysics, polarimetric-forward-operator 

SOURCE: Jacob Shpund, Alexander Khain, and Barry Lynn (The Hebrew University of Jerusalem)

DESCRIPTION OF CHANGES: 
A new updated Fast Spectral Bin Microphysics (FSBM) cloud microphysical scheme.

There are updates across all the FSBM microphysical scheme including: 
1. switch to use either graupel or hail, condensation/evaporation, nucleation (cloud base 
nucleation, 3 log-normal user-defined aerosol distribution), 
2. adaptive cond./evap. time-step, 
3. updated collision-coalescence, 
4. spontaneous rain breakup, 
5. spontaneous snow breakup.

A forward polarimetric operator is coupled to the FSBM scheme. The user can see the total 
reflectivity field, as well as the per hydrometeor total reflectivity (rain, snow, graupel/hail). 
Please see mandatory input tables info in the RELEASE NOTE.    

Modified PR from the original #848 and then the additionally modified PR #1085 
A new version of the FAST (33 bins) Spectral-Bin Microphysics (FSBM). 
From #848 -> #1085 
1. Inadvertent white space removed
2. Build with NMM
3. Remove -r8 option

From #1085 to #1097 
1. Fix conflict introduced with #1095 "Separating new_diagnostics from the rest in 
diag_misc module"

LIST OF MODIFIED FILES: 
M       Makefile
M       Registry/registry.em_shared_collection
A       Registry/registry.polrad
M       Registry/registry.sbm
M       arch/postamble
M       configure
M       dyn_em/solve_em.F
M       phys/Makefile
M       phys/module_diag_nwp.F
M       phys/module_diagnostics_driver.F
M       phys/module_microphysics_driver.F
A       phys/module_mp_SBM_polar_radar.F
M       phys/module_mp_fast_sbm.F
M       phys/module_physics_init.F
M       share/module_check_a_mundo.F

TESTS CONDUCTED: 
1. Automated jenkins regression test is all PASS.
2. The code has been compiled and used successfully for 10h of deep convection of the 20th May 2011 MC3E campaign test case (a manuscript consist of scheme details has been submitted for publication).
    * Compiled using Intel compiler 2019 (ifort version 19.0.3.199)
    * Configured using options 15 INTEL (ifort/icc,  dmpar) and nest option 1.
    * BC/IC used the NCEP FNL (Final) Operational Global Analysis data are on 1x1 degree grids  

- Domain setup and time step (5s per 1.3 km):
```
&domains
 time_step                           = 15,
 time_step_fract_num         = 0,
 time_step_fract_den           = 1,
 max_dom                             = 2,
 dx                                   = 4000, 1333.33,
 dy                                   = 4000, 1333.33,
 grid_id                            = 1,     2,   
 parent_id                        = 0,     1,  
 parent_grid_ratio             = 1,     3, 
 parent_time_step_ratio   = 1,     3, 
```

- In order to use the bin-wise polarimetric forward operator, a new flag must exist in the  <namelist.input> file:
```
&physics
sbm_diagnostics                     = 1,     1, 
```

- The FAST SBM code is not currently configure to work with default 8-byte reals. 

RELEASE NOTE:
Fast (33 bins) Spectral-bin Microphysics (FSBM):  in order to run the new FSBM scheme, users needs to download an external directory named "SBM_input_33" consist of mandatory input tables and place it in the 'run' directory. In case the coupled polarimetric forward operator is to be used (e.g., 'sbm_diagnostics = 1'), a second directory consist of scattering amplitudes named "SBM_scatter_amplit.tgz" needs to be placed in the 'run' directory. Both directories are compressed and can be downloaded at the following link: 
https://drive.google.com/drive/folders/1qxYyQwKI1wKQYasDUkQvVgHs11prLiqA?usp=sharing
(on a Linux machine, you may need to escape the `?` character with `\?`).
The FAST SBM code is not currently configure to work with default 8-byte reals.
@smileMchen smileMchen deleted the BEFORE_SPACE_REMOVAL branch January 15, 2021 02:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants