From 97132b5b7c8afdede410cb7c04064e178278d016 Mon Sep 17 00:00:00 2001 From: Dave Gill Date: Wed, 10 Nov 2021 23:57:50 +0000 Subject: [PATCH 1/2] Generalized method to handle different PBL options by domain TYPE: bug fix KEYWORDS: PBL, LES, proxy SOURCE: internal DESCRIPTION OF CHANGES: Problem: Only the YSU PBL scheme has no packaged variables. If a user wants to have any other PBL scheme on the CG, and an LES PBL on the FG, the model will have memory corruption problems. This error is hard to detect. Inside the model, this causes an inconsistent number of variables on the CG and FG, which then causes segmentation faults when trying to do feedback or advection of unavailable fields. Solution: Introduce a derived namelist entry (MAX_DOM sized). This proxy namelist value is used for space allocation due to packaging. The existing PBL physics definitions, and the associated packaged variables, have been replicated. The proxy entries are used to define the space, and the original entries are used for the automatically generated names. 1. There is no difference for the user facing namelist interface. 2. The developers see no differences inside of the WRF dynamics, or the physics schemes or drivers. 3. The only developer impact would be when adding additional packaged variables to an existing PBL scheme, or when introducing a new scheme. ISSUE: Fixes #1514 "MYNN in parent, LES in a nest: sometimes the sim does not run" LIST OF MODIFIED FILES: modified: Registry/Registry.EM_COMMON modified: share/module_check_a_mundo.F TESTS CONDUCTED: 1. A few test scenarios of the combinations of PBL options were tested. These results were printed out from the real program (same part of check_a_mundo that the WRF model sees). In each of the examples, the check_a_mundo logic translates the namelist entries into the required packages. . bl_pbl_physics = 5, 0, The memory allocation on domain 2 is set to the same packaged variables as on domain 1. ``` PBL PBL PROXY PHYSICS (mem) (scheme) ------------------ 5 5 5 0 ------------------ ``` . bl_pbl_physics = 1, 0, The memory allocation on domain 2 is set to the same packaged variables as on domain 1. ``` PBL PBL PROXY PHYSICS (mem) (scheme) ------------------ 1 1 1 0 ------------------ ``` . bl_pbl_physics = 1, 1, The memory allocation remains the default, package definitions for domains 1 and 2 remain unchanged. ``` PBL PBL PROXY PHYSICS (mem) (scheme) ------------------ 1 1 1 1 ------------------ ``` . bl_pbl_physics = 0, 0, The memory allocation remains the default, package definitions for domains 1 and 2 remain unchanged. ``` PBL PBL PROXY PHYSICS (mem) (scheme) ------------------ 0 0 0 0 ------------------ ``` 2. Hopefully, Jenkins tests are all PASS. RELEASE NOTE: A generalized method to handle domain-wise inconsistent PBL options has been introduced. Previously, this problem caused an inconsistent number of variables on the CG and FG, which caused segmentation faults when trying to do feedback or advection of unavailable fields. For users that tried the LES PBL option and a non-YSU PBL scheme (AND it worked!), it is possible that the results will be somewhat different due to the CG FG interactions during feedback and the generation of lateral boundary conditions for the nest. Scheme developers are pointed to the new packaging for the PBL schemes (look for the string `_proxy`). --- Registry/Registry.EM_COMMON | 40 +++++++++++++++++++++++++++--------- share/module_check_a_mundo.F | 31 ++++++++++++++++++++++++++++ 2 files changed, 61 insertions(+), 10 deletions(-) diff --git a/Registry/Registry.EM_COMMON b/Registry/Registry.EM_COMMON index 8b4e203bf2..82a6640412 100644 --- a/Registry/Registry.EM_COMMON +++ b/Registry/Registry.EM_COMMON @@ -2424,6 +2424,7 @@ rconfig integer irr_ph namelist,physics max_domains 0 rconfig integer sf_surface_physics namelist,physics max_domains -1 rh "sf_surface_physics" "" "" rconfig integer bl_pbl_physics namelist,physics max_domains -1 rh "bl_pbl_physics" "" "" +rconfig integer bl_pbl_physics_proxy derived max_domains -1 rh "bl_pbl_physics_proxy" "" "" rconfig integer bl_mynn_tkebudget namelist,physics max_domains 0 rh "bl_mynn_tkebudget" "" "" rconfig integer ysu_topdown_pblmix namelist,physics 1 1 rh "ysu_topdown_pblmix" "" "" rconfig integer shinhong_tke_diag namelist,physics max_domains 0 rh "shinhong_tke_diag" "" "" @@ -3086,19 +3087,38 @@ package drip sf_surf_irr_alloc==2 - state:irrigat package sprinkler sf_surf_irr_alloc==3 - state:irrigation,irr_rand_field +package les_proxy bl_pbl_physics_proxy==0 - - +package ysu_proxy bl_pbl_physics_proxy==1 - - +package myjpbl_proxy bl_pbl_physics_proxy==2 - state:tke_pbl,el_pbl +package gfs_proxy bl_pbl_physics_proxy==3 - - +package qnsepbl_proxy bl_pbl_physics_proxy==4 - state:tke_pbl,el_pbl,massflux_EDKF,entr_EDKF,detr_EDKF,thl_up,thv_up,rv_up,rt_up,rc_up,u_up,v_up,frac_up,rc_mf +package mynnpbl_proxy2 bl_pbl_physics_proxy==5 - scalar:qke_adv;state:qke,tke_pbl,sh3d,tsq,qsq,cov,el_pbl +package mynnpbl_proxy3 bl_pbl_physics_proxy==6 - scalar:qke_adv;state:qke,tke_pbl,sh3d,tsq,qsq,cov,el_pbl +package acmpbl_proxy bl_pbl_physics_proxy==7 - - +package boulac_proxy bl_pbl_physics_proxy==8 - state:el_pbl,tke_pbl,wu_tur,wv_tur,wt_tur,wq_tur +package camuwpbl_proxy bl_pbl_physics_proxy==9 - state:tauresx2d,tauresy2d,tpert2d,qpert2d,wpert2d,tke_pbl,smaw3d,wsedl3d,turbtype3d +package temfpbl_proxy bl_pbl_physics_proxy==10 - state:te_temf,kh_temf,km_temf,shf_temf,qf_temf,uw_temf,vw_temf,wupd_temf,mf_temf,thup_temf,qlup_temf,qtup_temf,cf3d_temf,hd_temf,lcl_temf,hct_temf,cfm_temf +package shinhong_proxy bl_pbl_physics_proxy==11 - state:el_pbl,tke_pbl +package gbmpbl_proxy bl_pbl_physics_proxy==12 - state:exch_tke,el_pbl,tke_pbl +package eeps_proxy bl_pbl_physics_proxy==16 - scalar:pek_adv,pep_adv;state:tke_pbl,pep_pbl +package mrf_proxy bl_pbl_physics_proxy==99 - - + +# Note - add packaged dependencies to the above "proxy" list. This enables +# LES PBL to be used on FG domains. +package lesscheme bl_pbl_physics==0 - - package ysuscheme bl_pbl_physics==1 - - -package myjpblscheme bl_pbl_physics==2 - state:tke_pbl,el_pbl +package myjpblscheme bl_pbl_physics==2 - - package gfsscheme bl_pbl_physics==3 - - -package qnsepblscheme bl_pbl_physics==4 - state:tke_pbl,el_pbl,massflux_EDKF,entr_EDKF,detr_EDKF,thl_up,thv_up,rv_up,rt_up,rc_up,u_up,v_up,frac_up,rc_mf -package mynnpblscheme2 bl_pbl_physics==5 - scalar:qke_adv;state:qke,tke_pbl,sh3d,tsq,qsq,cov,el_pbl -package mynnpblscheme3 bl_pbl_physics==6 - scalar:qke_adv;state:qke,tke_pbl,sh3d,tsq,qsq,cov,el_pbl +package qnsepblscheme bl_pbl_physics==4 - - +package mynnpblscheme2 bl_pbl_physics==5 - - +package mynnpblscheme3 bl_pbl_physics==6 - - package acmpblscheme bl_pbl_physics==7 - - -package boulacscheme bl_pbl_physics==8 - state:el_pbl,tke_pbl,wu_tur,wv_tur,wt_tur,wq_tur -package camuwpblscheme bl_pbl_physics==9 - state:tauresx2d,tauresy2d,tpert2d,qpert2d,wpert2d,tke_pbl,smaw3d,wsedl3d,turbtype3d -package temfpblscheme bl_pbl_physics==10 - state:te_temf,kh_temf,km_temf,shf_temf,qf_temf,uw_temf,vw_temf,wupd_temf,mf_temf,thup_temf,qlup_temf,qtup_temf,cf3d_temf,hd_temf,lcl_temf,hct_temf,cfm_temf -package shinhongscheme bl_pbl_physics==11 - state:el_pbl,tke_pbl -package gbmpblscheme bl_pbl_physics==12 - state:exch_tke,el_pbl,tke_pbl -package eepsscheme bl_pbl_physics==16 - scalar:pek_adv,pep_adv;state:tke_pbl,pep_pbl +package boulacscheme bl_pbl_physics==8 - - +package camuwpblscheme bl_pbl_physics==9 - - +package temfpblscheme bl_pbl_physics==10 - - +package shinhongscheme bl_pbl_physics==11 - - +package gbmpblscheme bl_pbl_physics==12 - - +package eepsscheme bl_pbl_physics==16 - - package mrfscheme bl_pbl_physics==99 - - package mynn_tkebudget bl_mynn_tkebudget==1 - state:qSHEAR,qBUOY,qDISS,qWT,dqke diff --git a/share/module_check_a_mundo.F b/share/module_check_a_mundo.F index 8acdd95e8f..81572dd1fc 100644 --- a/share/module_check_a_mundo.F +++ b/share/module_check_a_mundo.F @@ -406,6 +406,37 @@ END FUNCTION bep_bem_ngr_u END IF ENDDO +!----------------------------------------------------------------------- +! Handle the behind the scenes shuffling of the PBL package info, +! in support of generalizing usage of the LES PBL option on the FG. +!----------------------------------------------------------------------- + exists = .FALSE. + DO i = 1, model_config_rec % max_dom + IF ( .NOT. model_config_rec % grid_allowed(i) ) CYCLE + model_config_rec % bl_pbl_physics_proxy(i) = model_config_rec % bl_pbl_physics(i) + IF ( model_config_rec % bl_pbl_physics(i) .EQ. lesscheme ) THEN + exists = .TRUE. + END IF + ENDDO + IF ( ( exists ) .AND. ( model_config_rec % bl_pbl_physics(1) .NE. lesscheme ) ) THEN + DO i = 2, model_config_rec % max_dom + IF ( .NOT. model_config_rec % grid_allowed(i) ) CYCLE + IF ( model_config_rec % bl_pbl_physics(i) .EQ. lesscheme ) THEN + model_config_rec % bl_pbl_physics_proxy(i) = model_config_rec % bl_pbl_physics_proxy(1) + END IF + ENDDO + END IF +print *,' DAVE TEST' +print *,' PBL PBL ' +print *,' PROXY PHYSICS' +print *,' (mem) (scheme)' +print *,' ------------------' +DO i = 1, model_config_rec % max_dom +print *,model_config_rec % bl_pbl_physics_proxy(i),model_config_rec % bl_pbl_physics(i) +ENDDO +print *,' ------------------' +stop + !----------------------------------------------------------------------- ! Urban physics set up. If the run-time option for use_wudapt_lcz = 0, ! then the number of urban classes is 3. Else, if the use_wudapt_lcz = 1, From 07ab3cd0245fd0b5faafc2255b642c9cc664cbb6 Mon Sep 17 00:00:00 2001 From: Dave Gill Date: Thu, 11 Nov 2021 00:23:51 +0000 Subject: [PATCH 2/2] Zap the debug "stop" statement modified: share/module_check_a_mundo.F --- share/module_check_a_mundo.F | 1 - 1 file changed, 1 deletion(-) diff --git a/share/module_check_a_mundo.F b/share/module_check_a_mundo.F index 81572dd1fc..126c3bb822 100644 --- a/share/module_check_a_mundo.F +++ b/share/module_check_a_mundo.F @@ -435,7 +435,6 @@ END FUNCTION bep_bem_ngr_u print *,model_config_rec % bl_pbl_physics_proxy(i),model_config_rec % bl_pbl_physics(i) ENDDO print *,' ------------------' -stop !----------------------------------------------------------------------- ! Urban physics set up. If the run-time option for use_wudapt_lcz = 0,