Skip to content

CMAQ-WRF coupling: cherry pick #1609

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
weiwangncar merged 16 commits into from
Jan 26, 2022
Merged

CMAQ-WRF coupling: cherry pick #1609

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
de738e3
TYPE: new feature
dwongepa Mar 13, 2021
275c59d
fix compilation error with solve rho
davegill Dec 19, 2021
3cd5dcb
Fix inconsistent double precision complex functions
davegill Dec 19, 2021
1572dfa
small tweaks from developer (switched back to use lioapi_temp variabl…
dwongepa Jan 3, 2022
6ab6adb
Last minute cleanups
davegill Jan 19, 2022
bf8373c
Get rid of the string "twoway" in CMAQ file names
davegill Jan 20, 2022
355918e
Merge branch 'develop' into develop+cmaq_cherry_pick
davegill Jan 20, 2022
077bac6
Zap dupe file module_ra_rrtmg_sw_cmaq.F
davegill Jan 24, 2022
ed69f2a
CMAQ fix registry name 17:15
davegill Jan 25, 2022
f9d165c
TLADJ mods for CMAQ - arg from solve to rk part1
davegill Jan 25, 2022
92f657d
Play with optional args in TLADJ solve -> first rk part1
davegill Jan 25, 2022
63a6111
BF: NETCDFPAR "Error" in build log when option not used
davegill Jan 22, 2022
b55d990
Bug in the configure file, does not print out NC version
davegill Jan 22, 2022
213893d
NCPAR FIX nix DA 23:42
davegill Jan 23, 2022
8827d3e
modified arch/Config.pl to accommodate file name shortening in the WR…
dwongepa Jan 25, 2022
7ee699a
Replace nonstandard ISNAN() with IEEE_IS_NAN()
davegill Jan 25, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -988,6 +988,10 @@ physics :
@ echo '--------------------------------------'
if [ $(WRF_CHEM) -eq 0 ] ; then \
( cd phys ; $(MAKE) CF2=" " ) ; \
if [ $(WRF_CMAQ) -eq 1 ] ; then \
@ echo '----------- make cmaq ----------------' ; \
( rm -f main/libcmaqlib.a; cd cmaq ; $(MAKE) -f Makefile.twoway ) ; \
fi \
else \
( cd phys ; $(MAKE) CF2="$(CHEM_FILES2)" ) ; \
fi
Expand Down
8 changes: 8 additions & 0 deletions Registry/Registry.EM
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,14 @@
include registry.dimspec
include registry.em_shared_collection

# WRF-CMAQ coupled model
rconfig integer wrf_cmaq_option namelist,wrf_cmaq 1 0
rconfig integer wrf_cmaq_freq namelist,wrf_cmaq 1 1
rconfig integer met_file_tstep namelist,wrf_cmaq 1 10000

rconfig logical direct_sw_feedback namelist,wrf_cmaq 1 .false.
rconfig logical feedback_restart namelist,wrf_cmaq 1 .false.

@weiwangncar weiwangncar Dec 29, 2021

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@davegill @dwongepa Should these line be added to Registry.EM_COMMON?

@weiwangncar weiwangncar Jan 5, 2022

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@davegill Do you have a preference where these couple of new lines should be placed? It could be placed in the new registry file added: registry.WRF-CMAQ-twoway, could it?


# added to output 5 for ESMF
state real landmask ij misc 1 - i0125rh056d=(interp_fcnm_imask)u=(copy_fcnm) "LANDMASK" "LAND MASK (1 FOR LAND, 0 FOR WATER)" ""
state real lakemask ij misc 1 - i012rhd=(interp_fcnm_imask)u=(copy_fcnm) "LAKEMASK" "LAKE MASK (1 FOR LAKE, 0 FOR NON-LAKE)" ""
Expand Down
51 changes: 51 additions & 0 deletions Registry/registry.WRF-CMAQ-twoway
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
#state real mass_ws_i ikj twoway_feedback_data 1 - r "mass_ws_i" "Water soluble i mode" "ug/m**3"
#state real mass_ws_j ikj twoway_feedback_data 1 - r "mass_ws_j" "Water soluble j mode" "ug/m**3"
#state real mass_ws_k ikj twoway_feedback_data 1 - r "mass_ws_k" "Water soluble k mode" "ug/m**3"
#state real mass_in_i ikj twoway_feedback_data 1 - r "mass_in_i" "Water insoluble i mode" "ug/m**3"
#state real mass_in_j ikj twoway_feedback_data 1 - r "mass_in_j" "Water insoluble j mode" "ug/m**3"
#state real mass_in_k ikj twoway_feedback_data 1 - r "mass_in_k" "Water insoluble k mode" "ug/m**3"
#state real mass_ec_i ikj twoway_feedback_data 1 - r "mass_ec_i" "Elemental carbon i mode" "ug/m**3"
#state real mass_ec_j ikj twoway_feedback_data 1 - r "mass_ec_j" "Elemental carbon j mode" "ug/m**3"
#state real mass_ec_k ikj twoway_feedback_data 1 - r "mass_ec_k" "Elemental carbon k mode" "ug/m**3"
#state real mass_ss_i ikj twoway_feedback_data 1 - r "mass_ss_i" "Seasalt i mode" "ug/m**3"
#state real mass_ss_j ikj twoway_feedback_data 1 - r "mass_ss_j" "Seasalt j mode" "ug/m**3"
#state real mass_ss_k ikj twoway_feedback_data 1 - r "mass_ss_k" "Seasalt k mode" "ug/m**3"
#state real mass_h2o_i ikj twoway_feedback_data 1 - r "mass_h2o_i" "water i mode" "ug/m**3"
#state real mass_h2o_j ikj twoway_feedback_data 1 - r "mass_h2o_j" "water j mode" "ug/m**3"
#state real mass_h2o_k ikj twoway_feedback_data 1 - r "mass_h2o_k" "water k mode" "ug/m**3"
#state real dgn_i ikj twoway_feedback_data 1 - r "dgn_i" "diameter i mode" "m"
#state real dgn_j ikj twoway_feedback_data 1 - r "dgn_j" "diameter j mode" "m"
#state real dgn_k ikj twoway_feedback_data 1 - r "dgn_k" "diameter k mode" "m"
#state real sig_i ikj twoway_feedback_data 1 - r "sig_i" "standard deviations i mode" ""
#state real sig_j ikj twoway_feedback_data 1 - r "sig_j" "standard deviations j mode" ""
#state real sig_k ikj twoway_feedback_data 1 - r "sig_k" "standard deviations k mode" ""

#state real prev_rainnc ij twoway_feedback_data 1 - r "prev_rainnc" "previous accumlated rainnc" "mm"
#state real prev_rainc ij twoway_feedback_data 1 - r "prev_rainc" "previous accumlated rainc" "mm"

#state real ozone ikj twoway_feedback_data 1 - r "OZONE" "ozone value" ""

#state real sw_gtauxar_01 ikj misc 1 - hr "SW_GTAUXAR_01" "SW Optical depth of lamda = 0.388 um" ""
#state real sw_gtauxar_02 ikj misc 1 - hr "SW_GTAUXAR_02" "SW Optical depth of lamda = 0.533 um" ""
#state real sw_gtauxar_03 ikj misc 1 - hr "SW_GTAUXAR_03" "SW Optical depth of lamda = 0.702 um" ""
#state real sw_gtauxar_04 ikj misc 1 - hr "SW_GTAUXAR_04" "SW Optical depth of lamda = 1.010 um" ""
#state real sw_gtauxar_05 ikj misc 1 - hr "SW_GTAUXAR_05" "SW Optical depth of lamda = 1.271 um" ""
#state real sw_ttauxar_01 ij misc 1 - hr "SW_TTAUXAR_01" "Optical depth sum of SW_GTAUXAR_01" ""
#state real sw_ttauxar_02 ij misc 1 - hr "SW_TTAUXAR_02" "Optical depth sum of SW_GTAUXAR_02" ""
#state real sw_ttauxar_03 ij misc 1 - hr "SW_TTAUXAR_03" "Optical depth sum of SW_GTAUXAR_03" ""
#state real sw_ttauxar_04 ij misc 1 - hr "SW_TTAUXAR_04" "Optical depth sum of SW_GTAUXAR_04" ""
#state real sw_ttauxar_05 ij misc 1 - hr "SW_TTAUXAR_05" "Optical depth sum of SW_GTAUXAR_05" ""
#state real sw_asy_fac_01 ikj misc 1 - hr "SW_ASY_FAC_01" "Corresponding Asymmetry Factor of SW_GTAUXAR_01" ""
#state real sw_asy_fac_02 ikj misc 1 - hr "SW_ASY_FAC_02" "Corresponding Asymmetry Factor of SW_GTAUXAR_02" ""
#state real sw_asy_fac_03 ikj misc 1 - hr "SW_ASY_FAC_03" "Corresponding Asymmetry Factor of SW_GTAUXAR_03" ""
#state real sw_asy_fac_04 ikj misc 1 - hr "SW_ASY_FAC_04" "Corresponding Asymmetry Factor of SW_GTAUXAR_04" ""
#state real sw_asy_fac_05 ikj misc 1 - hr "SW_ASY_FAC_05" "Corresponding Asymmetry Factor of SW_GTAUXAR_05" ""
#state real sw_ssa_01 ikj misc 1 - hr "SW_SSA_01" "Corresponding Single scattering Albedo of SW_GTAUXAR_01" ""
#state real sw_ssa_02 ikj misc 1 - hr "SW_SSA_02" "Corresponding Single scattering Albedo of SW_GTAUXAR_02" ""
#state real sw_ssa_03 ikj misc 1 - hr "SW_SSA_03" "Corresponding Single scattering Albedo of SW_GTAUXAR_03" ""
#state real sw_ssa_04 ikj misc 1 - hr "SW_SSA_04" "Corresponding Single scattering Albedo of SW_GTAUXAR_04" ""
#state real sw_ssa_05 ikj misc 1 - hr "SW_SSA_05" "Corresponding Single scattering Albedo of SW_GTAUXAR_05" ""

#state real sw_zbbcddir ij misc 1 - hr "SW_ZBBCDDIR" "Clear sky downward direct shortwave flux" "W m-2"
#state real sw_dirdflux ij misc 1 - hr "SW_DIRDFLUX" "Direct downward shortwave surface flux" "W m-2"
#state real sw_difdflux ij misc 1 - hr "SW_DIFDFLUX" "Diffuse downward shortwave surface flux" "W m-2"
1 change: 1 addition & 0 deletions Registry/registry.em_shared_collection
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,3 +28,4 @@ include registry.new3d_wif
include registry.trad_fields
include registry.solar_fields
include registry.diags
include registry.WRF-CMAQ-twoway
155 changes: 154 additions & 1 deletion arch/Config.pl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,9 @@
# Be sure to run as ./configure (to avoid getting a system configure command by mistake)
#

use Cwd qw(getcwd);
$wrf_cmaq_option = $ENV{'WRF_CMAQ'}; # determine building WRF-CMAQ coupled model or not

select((select(STDOUT), $|=1)[0]);
$sw_perl_path = perl ;
$sw_netcdf_path = "" ;
Expand Down Expand Up @@ -444,11 +447,157 @@
$latchon = 0 ;
while ( <CONFIGURE_DEFAULTS> )
{
if ( $_ =~ /ifort compiler/ )
{ $lioapi = 'Linux2_x86_64ifort';
}
elsif ( $_ =~ /PGI compiler/ )
{ $lioapi = 'Linux2_x86_64pg';
}
elsif ( $_ =~ /gfortran compiler/ )
{ $lioapi = 'Linux2_x86_64gfort';
}

if ( substr( $_, 0, 5 ) eq "#ARCH" && $latchon == 1 )
{
close CONFIGURE_DEFAULTS ;
if ( $sw_opt_level eq "-f" ) {
open CONFIGURE_DEFAULTS, "cat ./arch/postamble ./arch/noopt_exceptions_f |" or die "horribly" ;

# determine whether variable EM_MODULE_DIR contains -I../cmaq or not
my $file = "Makefile";
open(FH, $file) or die("File $file not found");

$lib_path_wo_cmaq = 1;
while ( my $String = <FH> )
{ if($String =~ /-I..\/dyn_em -I..\/cmaq/)
{ $lib_path_wo_cmaq = 0;
}
}
close (FH);

# determine whether declarations in Registry/registry.WRF-CMAQ-twoway is commented out or not
my $file = "Registry/registry.WRF-CMAQ-twoway";
open (FH, $file) or die("File $file not found");

$registry_wo_cmaq = 0;
while ( my $String = <FH> )
{ if($String =~ /#state/)
{ $registry_wo_cmaq = 1;
}
}
close (FH);

# determine whether express #NOWIN LIB_BUNDLED contains $(IOAPI_LIB) or not
my $file = "arch/preamble";
open (FH, $file) or die("File $file not found");

$bundle_wo_ioapi = 1;
while ( my $String = <FH> )
{ if ( $String =~ /IOAPI_LIB/ )
{ $bundle_wo_ioapi = 0;
}
}
close (FH);

if ( $wrf_cmaq_option eq 1 ) # build WRF-CMAQ coupled model
{ system("cp Registry/registry.WRF-CMAQ-twoway_full Registry/registry.WRF-CMAQ-twoway");

if ( $lib_path_wo_cmaq == 1 )
{ open (FILE, "<Makefile") || die "File not found";
my @lines = <FILE>;
close (FILE);

foreach ( @lines )
{ $_ =~ s/ -I..\/dyn_em/ -I..\/dyn_em -I..\/cmaq/g;
}

open (FILE, ">Makefile") || die "File not found";
print FILE @lines;
close (FILE);
}

if ( $registry_wo_cmaq == 1 )
{ open (FILE, "<Registry/registry.WRF-CMAQ-twoway") || die "File not found";
my @lines = <FILE>;
close (FILE);

foreach (@lines)
{ $_ =~ s/#state/state/g;
}

open (FILE, ">Registry/registry.WRF-CMAQ-twoway") || die "File not found";
print FILE @lines;
close (FILE);
}

if ( $bundle_wo_ioapi == 1 )
{ open (FILE, "<arch/preamble") || die "File not found";
my @lines = <FILE>;
close (FILE);

foreach (@lines)
{ $_ =~ s/#NOWIN LIB_BUNDLED = \\/#NOWIN LIB_BUNDLED = \$(IOAPI_LIB) \\/g;
}

open (FILE, ">arch/preamble") || die "File not found";
print FILE @lines;
close (FILE);
}

open (FH, '>', wrf_cmaq_path) or die $! ;
$ioapi_path = $ENV{'IOAPI'} ;
print FH "CMAQLIB = libcmaqlib.a \n" ;
print FH "IOAPI = $ioapi_path\n" ;
print FH "LIOAPI = $lioapi\n" ;
print FH "IOAPI_LIB = -L$ioapi_path/$lioapi -lioapi \n" ;
close (FH) ;
open CONFIGURE_DEFAULTS, "cat wrf_cmaq_path ./arch/postamble ./arch/noopt_exceptions_f |" or die "horribly" ;
}
else
{ if ( $lib_path_wo_cmaq == 0 )
{ open (FILE, "<Makefile") || die "File not found";
my @lines = <FILE>;
close (FILE);

foreach (@lines)
{ $_ =~ s/ -I..\/cmaq//g;
}

open (FILE, ">Makefile") || die "File not found";
print FILE @lines;
close (FILE);

}

if ( $registry_wo_cmaq == 0 )
{ open (FILE, "<Registry/registry.WRF-CMAQ-twoway") || die "File not found";
my @lines = <FILE>;
close (FILE);

foreach(@lines)
{ $_ =~ s/state/#state/g;
}

open (FILE, ">Registry/registry.WRF-CMAQ-twoway") || die "File not found";
print FILE @lines;
close (FILE);
}

if ( $bundle_wo_ioapi == 0 )
{ open (FILE, "<arch/preamble") || die "File not found";
my @lines = <FILE>;
close (FILE);

foreach (@lines)
{ $_ =~ s/ \$\(IOAPI_LIB\)//g;
}

open (FILE, ">arch/preamble") || die "File not found";
print FILE @lines;
close (FILE);
}

open CONFIGURE_DEFAULTS, "cat ./arch/postamble ./arch/noopt_exceptions_f |" or die "horribly" ;
}
} else {
open CONFIGURE_DEFAULTS, "cat ./arch/postamble ./arch/noopt_exceptions |" or die "horribly" ;
}
Expand Down Expand Up @@ -777,6 +926,10 @@
close POSTAMBLE ;
close ARCH_NOOPT_EXCEPTIONS ;

if ( $wrf_cmaq_option eq 1 )
{ unlink "wrf_cmaq_path";
}

open CONFIGURE_WRF, "> configure.wrf" or die "cannot append configure.wrf" ;
open ARCH_PREAMBLE, "< arch/preamble" or die "cannot open arch/preamble" ;
my @preamble;
Expand Down
8 changes: 6 additions & 2 deletions clean
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,15 @@
set nonomatch


foreach dir ( frame chem share dyn_em phys main tools wrftladj )
foreach dir ( frame chem share dyn_em phys cmaq main tools wrftladj )
if ( -d $dir ) then
( cd $dir ; echo $dir ; /bin/rm -f core wrf *.f90 *.exe *.kmo *.mod *.o *.obj *.inc *.F90 *.a \
if ( $dir == cmaq ) then
( cd $dir ; echo $dir ; /bin/rm -f *.o *.mod ) >& /dev/null
else
( cd $dir ; echo $dir ; /bin/rm -f core wrf *.f90 *.exe *.kmo *.mod *.o *.obj *.inc *.F90 *.a \
db_* Warnings module_state_description.F module_dm.F gmeta \
wrfdata whatiread rsl.* show_domain* ) >& /dev/null
endif
endif
end

Expand Down
7 changes: 7 additions & 0 deletions configure
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ while [ $# -ge 1 ]; do
-r8) rword="-r8" ;;
-time) shift ; FORTRAN_COMPILER_TIMER=$1 ;;
chem) WRF_CHEM=1 ;;
cmaq) WRF_CMAQ=1 ;;
kpp) WRF_KPP=1 ;;
radardfi) WRF_DFI_RADAR=1 ;;
wrfda) wrf_core=DA_CORE ;;
Expand Down Expand Up @@ -460,6 +461,12 @@ if [ -n "$WRF_DFI_RADAR" ] ; then
compileflags="${compileflags}!-DWRF_DFI_RADAR=1"
fi
fi

if [ -n "$WRF_CMAQ" ] ; then
echo building WRF with CMAQ option
compileflags="${compileflags}!-DWRF_CMAQ"
fi

if [ -n "$WRF_CHEM" ] ; then
if [ $WRF_CHEM = 1 ] ; then
echo building WRF with chemistry option
Expand Down
Loading