From e0a374335b339a2f03cce5d72338a0cb9f9a4fd7 Mon Sep 17 00:00:00 2001 From: michelleharrold Date: Wed, 15 Sep 2021 20:54:28 -0600 Subject: [PATCH 01/16] Add METplus-based ensemble verification (#575) * Fix to post flat file. * Create MET and METplus config files under ush/templates/parm * Added script to pull and reorg ccpa data. Added a script to run gridstat with METplus. Updated MET and METplus config files. * Added new jjob for running grid-stat vx. Updated setup.sh to include grid-stat vx. Updated run_gridstatvx script. * Fixed typo on script name from ksh to sh * Moved some hard coded items out from the script to the XML * Updates to get METplus to run with fewer hard-coded paths. * Updates to add grid-stat task to XML generation. * Bug fixes for adding grid-stat to XML generation * Updates to remove hard-coded paths in config files * Change log dir to put master_metplus log file with other logs under log/, rather than default logs/. * Updates to generate xml without hard-coded paths for MET * Add hera gridstat module file * Add METplus point-stat task for both sfc and upper air * Small tweaks to remove hard coded paths and add some flexibility * Updates for adding point-stat into auto-generated xml * Add in function to set point-stat task to FALSE * Final tweaks to get it to generate the xml correctly * Minor updates to run ensure 0,6,12,18 * Tweaks to var list for Point-Stat * Add METplus settings to config_defaults * Move quote for end of settings and fix extra comment. * Fix typos to populate templates correctly * Updated to include SCRIPTSDIR and other MET specific settings along with updates to FHR syntax * Update module loads on hera * Fixed comment for BOTH_VARn_THRESH to avoid syntax issues * Added files to run grid_stat for a variety of accumulation intervals, including 3, 6, and 24h * Added module load hpss * Remove module load informatino from these scripts * Updated the method of turning on/off vx tasks using jinja template if statement * Remove commented out lines of code. Fixed typo. Removed gen_wflow.out file. * Updated pull scripts to have file names dependent on date to pull from HPSS. Updated to export a few more local variables that METplus conf needed in scripts. Updated workflow to use service queue (for now) to for 1h grid_stat and point_stat run and default for 3+h accumulation grid_stat runs) * moved common_hera.conf to common.conf - no platform specific information included that needs to be handled. * Remove common_hera.conf * Add scripts to pull and process MRMS data from NOAA HPSS * Updates for REFC vx tasks * updates to obs pull scripts * Update for adding in reflectivity verification using MRMS analyses and updating name of model output to RRFS rather than HRRR * Updates to account for CCPA issues on HPSS - day off for 00-05 UTC directories * Verification mods to feature/add metplus (#1) * Remove unused/outdated code (#313) ## DESCRIPTION OF CHANGES: * In setup.sh and generate_FV3LAM_wflow.sh, remove temporary codes that fix bugs in the FV3_GFS_2017_gfdlmp_regional suite definition file because those bugs have been fixed (in the ufs-weather-model repo). * In setup.sh, remove block of code that is no longer necessary because chgres_cube can now initialize from external model data with either 4 or 9 soil levels, and run with LSMs of either 4 or 9 soil levels. * Remove modifications to LD_LIBRARY_PATH in exregional_run_fcst.sh. * For the make_ics and make_lbcs tasks, move the setting of APRUN and other machine-specific actions from the J-job to the ex-script in order to be consistent with the other workflow tasks. * Fix indentation and edit comments. * Remove unused file load_fv3gfs_modules.sh. ## TESTS CONDUCTED: Ran two WE2E tests on hera, new_ESGgrid and new_GFDLgrid: * new_ESGgrid uses the FV3_GFS_2017_gfdlmp_regional suite. The test was successful. * new_GFDLgrid uses the FV3_GFS_2017_gfdlmp suite. The test was successful. ## ISSUE (optional): This resolves issue #198. * Add and call a function that checks for use of Thompson microphysics parameterization in the SDF and if so, adjusts certain workflow arrays to contain the names and other associated values of the fixed files needed by this parameterization so that those files are automatically copied and/or linked to. (#319) ## DESCRIPTION OF CHANGES: Add and call a function that checks for use of Thompson microphysics parameterization in the suite definition file (SDF). If not, do nothing. If so, add to the appropriate workflow arrays the names and other associated values of the fixed files needed by this parameterization so that they are automatically copied and/or linked to instead of being regenerated from scratch in the run_fcst task. ## TESTS CONDUCTED: On hera, ran two WE2E tests, one in NCO mode (nco_RRFS_CONUS_25km_HRRRX_RAPX) and the other in community mode (suite_FV3_GSD_v0). These use suites FV3_GSD_SAR and FV3_GSD_v0, respectively, and both of these call Thompson microphysics. Both succeeded. ## ISSUE (optional): This PR resolves issue #297. * RRFS_v1beta SDF changes after reverting from GSL to GFS GWD suite (#322) (#327) ## DESCRIPTION OF CHANGES: Removed checks on the RRFS_v1beta SDF implemented for use with the GSL GWD suite (now uses the GFS GWD suite). No longer copies staged orography files necessary for the GSL GWD suite. ## TESTS CONDUCTED: Runs to completion on Hera. End-to-end runs DOT_OR_USCORE and suite_FV3_RRFS_v1beta succeeded on Cheyenne. Co-authored-by: JeffBeck-NOAA <55201531+JeffBeck-NOAA@users.noreply.github.com> * Update FV3.input.nml for fhzero = 1.0 * Updated conf files for file name conventions. * Updated MET scripts and MRMS pull scripts. * Adjust RRFS_CONUS_... grids (#294) ## DESCRIPTION OF CHANGES: * Adjust RRFS_CONUS_25km, RRFS_CONUS_13km, and RRFS_CONUS_3km grid parameters so that: * All grids, including their 4-cell-wide halos, lie completely within the HRRRX domain. * All grids have dimensions nx and ny that factor "nicely", i.e. they don't have factors greather than 7. * The write-component grids corresponding to these three native grids cover as much of the native grids as possible without going outside of the native grid boundaries. The updated NCL scripts (see below) were used to generate the write-component grid parameters. * For the RRFS_CONUS_13km grid, reduce the time step (DT_ATMOS) from 180sec to 45sec. This is necessary to get a successful forecast with the GSD_SAR suite, and thus likely also the RRFS_v1beta suite. * Modify WE2E testing system as follows: * Add new tests with the RRFS_CONUS_25km, RRFS_CONUS_13km, and RRFS_CONUS_3km grids that use the GFS_v15p2 and RRFS_v1beta suites (which are now the ones officially supported in the first release of the short-range weather app) instead of the GFS_v16beta and GSD_SAR suites, respectively. * For clarity, rename the test configuration files that use the GFS_v16beta and GSD_SAR suites so they include the suite name. * Update list of WE2E tests (baselines_list.txt). * Update the NCL plotting scripts to be able to plot grids with the latest version of the workflow. ## TESTS CONDUCTED: On hera, ran tests with all three grids with the GFS_v15p2 and RRFS_v1beta suites (a total of 6 tests). All were successful. * Remove redundant model_configure.${CCPP_PHYS_SUITE} template files; use Jinja2 to create model_configure (#321) ## DESCRIPTION OF CHANGES: * Remove model_configure template files whose names depend on the physics suite, i.e. files with names of the form model_configure.${CCPP_PHYS_SUITE}. Only a single template file is needed because the contents of the model_configure file are not suite dependent. This leaves just one template file (named model_configure). * Change the function create_model_configure_file.sh and the template file model_configure so they use jinja2 instead of sed to replace placeholder values. * Absorb the contents of the write-component template files wrtcmp_lambert_conformal, wrtcmp_regional_latlon, and wrtcmp_rotated_latlon into the new jinja2-compliant model_configure file. We can do this because Jinja2 allows use of if-statements in the template file. * In the new model_configure jinja2 template file, include comments to explain the various write-component parameters. ## TESTS CONDUCTED: On Hera, ran the two WE2E tests new_ESGgrid and new_GFDLgrid. The first uses a "lambert_conformal" type of write-component grid, and the second uses a "rotated_latlon" type of write-component grid. (The write-component also allows "regional_latlon" type grids, which is just the usual earth-relative latlon coordinate system, but we do not have any cases that use that.) Both tests succeeded. ## ISSUE (optional): This PR resolves issue #281. * Add Thompson ice- and water-friendly aerosol climo file support (#332) * Add if statement in set_thompson_mp_fix_files.sh to source Thompson climo file when using a combination of a Thompson-based SDF and non-RAP/HRRR external model data * Modify if statement based on external models for Thompson climo file * Remove workflow variable EMC_GRID_NAME (#333) ## DESCRIPTION OF CHANGES: * Remove the workflow variable EMC_GRID_NAME. Henceforth, PREDEF_GRID_NAME is the only variable that can be used to set the name of the predefined grid to use. * Make appropriate change of variable name (EMC_GRID_NAME --> PREDEF_GRID_NAME) in the WE2E test configuration files. * Change anywhere the "conus" and "conus_c96" grids are specified to "EMC_CONUS_3km" and "EMC_CONUS_coarse", respectively. * Rename WE2E test configuration files with names containing the strings "conus" and "conus_c96" by replacing these strings with "EMC_CONUS_3km" and "EMC_CONUS_coarse", respectively. * Update the list of WE2E test names (tests/baselines_list.txt). * Bug fixes not directly related to grids: * In config.nco.sh, remove settings of QUEUE_DEFAULT, QUEUE_HPSS, and QUEUE_FCST since these are now set automatically (due to another PR). * In the template file FV3LAM_wflow.xml, add the ensemble member name after RUN_FCST_TN in the dependency of the run_post metatask. ## TESTS CONDUCTED: Since this change only affects runs in NCO mode, the following NCO-mode WE2E tests were rerun on hera, all successfully: ``` nco_EMC_CONUS_3km SUCCESS nco_EMC_CONUS_coarse SUCCESS nco_EMC_CONUS_coarse__suite_FV3_GFS_2017_gfdlmp SUCCESS nco_RRFS_CONUS_25km_HRRRX_RAPX SUCCESS nco_RRFS_CONUS_3km_FV3GFS_FV3GFS SUCCESS nco_RRFS_CONUS_3km_HRRRX_RAPX SUCCESS nco_ensemble SUCCESS ``` * Port workflow to Orion (#309) ## DESCRIPTION OF CHANGES: * Add stanzas for Orion where necessary. * Add new module files for Orion. * On Orion, both the slurm partition and the slurm QOS need to be specified in the rocoto XML in order to be able to have wall times longer than 30 mins (the partition needs to be specified because it is by default "debug", which has a limit of 30 mins). Thus, introduce modifications to more easily specify slurm partitions: * Remove the workflow variables QUEUE_DEFAULT_TAG, QUEUE_HPSS_TAG, and QUEUE_FCST_TAG that are currently used to determine whether QUEUE_DEFAULT, QUEUE_HPSS, and QUEUE_FCST specify the names of queue/QOS's or slurm partitions. * Add the workflow variables PARTITION_DEFAULT_TAG, PARTITION_HPSS_TAG, and PARTITION_FCST_TAG. These will be used to specify slurm partitions only, and the variables QUEUE_DEFAULT, QUEUE_HPSS, and QUEUE_FCST will be used to specify queues/QOS's only. IMPORTANT NOTE: On Orion, in order to load the regional_workflow environment needed for generating an experiment, the user must first issue the following commands: ``` module use -a /apps/contrib/miniconda3-noaa-gsl/modulefiles module load miniconda3 conda activate regional_workflow ``` ## TESTS CONDUCTED: Ran 11 WE2E tests on Orion, Hera, and Cheyenne. Results on Orion: ``` community_ensemble_2mems SUCCESS DOT_OR_USCORE SUCCESS grid_GSD_HRRR_AK_50km FAILURE - In the run_fcst task. * Error message: !!! (1) Error in subr radiation_aerosols: unrealistic surface pressure = 1 NaN new_ESGgrid SUCCESS new_GFDLgrid SUCCESS regional_001 SUCCESS regional_002 SUCCESS suite_FV3_GFS_v15p2 SUCCESS suite_FV3_GFS_v16beta SUCCESS suite_FV3_GSD_SAR SUCCESS suite_FV3_GSD_v0 SUCCESS ``` Results on Hera: ``` community_ensemble_2mems SUCCESS DOT_OR_USCORE SUCCESS grid_GSD_HRRR_AK_50km SUCCESS new_ESGgrid SUCCESS new_GFDLgrid SUCCESS regional_001 SUCCESS regional_002 SUCCESS suite_FV3_GFS_v15p2 SUCCESS suite_FV3_GFS_v16beta SUCCESS suite_FV3_GSD_SAR SUCCESS suite_FV3_GSD_v0 SUCCESS ``` Results on Cheyenne: ``` community_ensemble_2mems SUCCESS DOT_OR_USCORE SUCCESS grid_GSD_HRRR_AK_50km FAILURE - In run_fcst task. * Error message: !!! (1) Error in subr radiation_aerosols: unrealistic surface pressure = 1 NaN new_ESGgrid SUCCESS new_GFDLgrid SUCCESS regional_001 SUCCESS regional_002 SUCCESS suite_FV3_GFS_v15p2 SUCCESS suite_FV3_GFS_v16beta SUCCESS suite_FV3_GSD_SAR SUCCESS suite_FV3_GSD_v0 SUCCESS ``` All succeed except GSD_HRRR_AK_50km on Orion and Cheyenne. It is not clear why grid_GSD_HRRR_AK_50km fails on Orion and Cheyenne but not Hera. Seems to point to a bug in the forecast model. These two failures are not so important since this grid will soon be deprecated. Also tested successfully on Jet by @JeffBeck-NOAA and on Odin and Stampede by @ywangwof. ## ISSUE: This resolves Issue #152. ## CONTRIBUTORS: @JeffBeck-NOAA @ywangwof @christinaholtNOAA * Removed comments from exregional_get_mrms_files.sh and removed fhzero from FV3.input.yml * Update FV3.input.nml for fhzero = 1.0 * Updated conf files for file name conventions. * Updated MET scripts and MRMS pull scripts. * Removed comments from exregional_get_mrms_files.sh and removed fhzero from FV3.input.yml Co-authored-by: gsketefian <31046882+gsketefian@users.noreply.github.com> Co-authored-by: Michael Kavulich Co-authored-by: JeffBeck-NOAA <55201531+JeffBeck-NOAA@users.noreply.github.com> Co-authored-by: Jamie Wolff * Change cov_thresh for REFL to be a true max in nbrhood as SPC does. * Job script for get_obs_ccpa * Jobs script for get_obs_mrms * Jobs script for get_obs_ndas * Added external variables necessary to get_ccpa script * Updated workflow template with separate get obs tasks * Separated pull scripts from run scripts * Added necessary defaults/values for defining pull tasks * Added module files, default config.sh options, and changed dependencies for vx tasks * Changed name of new workflow to FV3LAM_wflow.xml * Added task get_obs_tn, removed config.sh, updated config_defaults and config.community.sh * Adjusted the community and default config files based on comments * Updated FV3LAM workflow * Fixed discrepancies in config.community.sh * Fixed discrepancies in config_defaults.sh * Fixed discrepancies in config_defaults.sh round 2 * Fixed discrepancies in config_defaults.sh round 3 * Fixed discrepancies in config_defaults.sh round 4 * Fixed discrepancies in config.community.sh round 2 * Fixed discrepancies in config.community.sh round 3 * Fixed discrepancies in generate_FV3LAM_wflow.sh * Fixed discrepancies in generate_FV3LAM_wflow.sh round 2 * Fixed discrepancies in generate_FV3LAM_wflow.sh round 3 * Updated FV3LAM_wflow template * Separated Pull Data Scripts from Run Vx Scripts: Feature/add_metplus (#2) * Job script for get_obs_ccpa * Jobs script for get_obs_mrms * Jobs script for get_obs_ndas * Added external variables necessary to get_ccpa script * Updated workflow template with separate get obs tasks * Separated pull scripts from run scripts * Added necessary defaults/values for defining pull tasks * Added module files, default config.sh options, and changed dependencies for vx tasks * Changed name of new workflow to FV3LAM_wflow.xml * Added task get_obs_tn, removed config.sh, updated config_defaults and config.community.sh * Adjusted the community and default config files based on comments * Updated FV3LAM workflow * Fixed discrepancies in config.community.sh * Fixed discrepancies in config_defaults.sh * Fixed discrepancies in config_defaults.sh round 2 * Fixed discrepancies in config_defaults.sh round 3 * Fixed discrepancies in config_defaults.sh round 4 * Fixed discrepancies in config.community.sh round 2 * Fixed discrepancies in config.community.sh round 3 * Fixed discrepancies in generate_FV3LAM_wflow.sh * Fixed discrepancies in generate_FV3LAM_wflow.sh round 2 * Fixed discrepancies in generate_FV3LAM_wflow.sh round 3 * Updated FV3LAM_wflow template * Fixed the dependencies of the vx tasks * Fixed Vx Task Dependencies in Workflow: Feature/add metplus (#3) * Job script for get_obs_ccpa * Jobs script for get_obs_mrms * Jobs script for get_obs_ndas * Added external variables necessary to get_ccpa script * Updated workflow template with separate get obs tasks * Separated pull scripts from run scripts * Added necessary defaults/values for defining pull tasks * Added module files, default config.sh options, and changed dependencies for vx tasks * Changed name of new workflow to FV3LAM_wflow.xml * Added task get_obs_tn, removed config.sh, updated config_defaults and config.community.sh * Adjusted the community and default config files based on comments * Updated FV3LAM workflow * Fixed discrepancies in config.community.sh * Fixed discrepancies in config_defaults.sh * Fixed discrepancies in config_defaults.sh round 2 * Fixed discrepancies in config_defaults.sh round 3 * Fixed discrepancies in config_defaults.sh round 4 * Fixed discrepancies in config.community.sh round 2 * Fixed discrepancies in config.community.sh round 3 * Fixed discrepancies in generate_FV3LAM_wflow.sh * Fixed discrepancies in generate_FV3LAM_wflow.sh round 2 * Fixed discrepancies in generate_FV3LAM_wflow.sh round 3 * Updated FV3LAM_wflow template * Fixed the dependencies of the vx tasks * Manual merge with develop that didn't seem to work before. Trying to get feature branch updated so it will run again! * Add local module files * Add environment variable for SCRIPTSDIR * Remove echo statement * Remove old module files * Update to config_default for walltime for ndas pull. Update to metplus parm for obs file template. Update to FV3LAM xml to not include 00 hour for verification * Update template to remove full path * Verification channges for obs. (#4) * Verification channges for obs. * Update config_defaults.sh for vx description * Update config_defaults.sh to remove extraneous MET info. Co-authored-by: Michelle Harrold * Initial METplus .confs and MET config files for EnsembleStat APCP * J-Job script for running ensemble stat * Exregional script for ensemble-stat * Added EnsembleStat.conf for A6 and A24. Added PCPCombine to A3, A6, and A24. * Added EnsembleStatConfig files for 6 and 24h * Copy of workflow template with precipitation ensemble tasks added. Will become main template when testing is complete * Added export statement for number of ensemble members * Added necessary task definitions in ush * Updated workflow to included ENTITY definitions for ensstat * Fixed typo * Added ens vx configs * Pull in updates from develop that were not merging properly. Small change to config.community to turn off vx tasks by default. * Added/mod files for point ens vx. * Updated metplus conf files for ens point vx * Did manual merge of these files because it was not handled properly automatically * Adding additional variables to METplus for regional workflow (#5) * Changes made based on meeting with Michelle and Jamie * Updating fork * Cleanup after merge * Added additional ens vx * Ensemble point vx mods * Additional updates for ens and det vx * ensgrid_mean and ensgrid_prob .conf files for APCP * Updates for ensemble vx. * Added mean and prob point-stat configs * Updates to ensgrid_vx * Updates for mean/prob vx. * Updates to FV3LAM_wflow.xml * Deterministic and ensembel vx updates. * Ensgrid mean * Update setup.sh * Changed workflow template title * Updates to deterministic and ensemble verification * Created EnsembleStat METplus conf and MET config files for REFC * Added reflectivity mean and prob METplus and MET config files. Updated APCP mean and prob METplus and MET config files. * Added all J-job scripts, exregional scripts, and necessary definitons for workflow generation for all ensgrid_mean and ensgrid_prob tasks * Updates to workflow to add ensgrid_vx * Changes I made to account for runtime errors. * Made changes to directory structures * Made changes to directory structures and variables * Changed log files and stage dir. * Changes for grid- and point-vx. * Updated METplus ensemble precip conf files. * Mods for ensemble and deterministic vx. * Change to GridStatConfig_REFC_mean * Updated EnsembleStat_REFC.conf * Updated to METv10.0.0 * Updated conf files for paths. * Updated FV3LAM_wflow.xml template. * Mods for vx dependencies * Updated for censor thresh in METplus conf files; changes to FV3LAM_wflow.xml after sync with develop. * Updated exregional_run_fcst.sh generate_FV3LAM_wflow.sh to address merge with develop. * Mods for ensemble precip vx, handling padded/non-padded ensemble member names, fixes for python environment for obs pull. * Changes to RETOP (units) and REFC (naming and level) verification. * Fix OUTPUT_BASE for deterministic vx. * Changes to some verification ex-scripts for syntax and path fixes. Included start end dates of incorrect 01-h CCPA data. Removed some extra lines in FV3LAM_wflow.xml template. * Changed comp. ref. variable name in GridStat_REFC_prob.conf * Changed comp. ref. level in GridStat_REFC_prob.conf * Updated logic for number padding in the directory name when running in ensemble mode. * Added MET ensemble vx WE2E test. * Modified location of obs to live outside cycle dir, allowing for obs to be shared across cycles. * Mods to address comments on PR575. * Updated ensemble METPlus conf files for changes to post output name. * Addessed comments in PR and mods for 10-m WIND. * Addressing final comments in PR. Co-authored-by: Jamie Wolff Co-authored-by: gsketefian <31046882+gsketefian@users.noreply.github.com> Co-authored-by: Michael Kavulich Co-authored-by: JeffBeck-NOAA <55201531+JeffBeck-NOAA@users.noreply.github.com> Co-authored-by: lindsayrblank Co-authored-by: Michelle Harrold Co-authored-by: PerryShafran-NOAA <62255233+PerryShafran-NOAA@users.noreply.github.com> --- jobs/JREGIONAL_RUN_VX_ENSGRID | 90 ++ jobs/JREGIONAL_RUN_VX_ENSGRID_MEAN | 90 ++ jobs/JREGIONAL_RUN_VX_ENSGRID_PROB | 90 ++ jobs/JREGIONAL_RUN_VX_ENSPOINT | 88 ++ jobs/JREGIONAL_RUN_VX_ENSPOINT_MEAN | 88 ++ jobs/JREGIONAL_RUN_VX_ENSPOINT_PROB | 88 ++ jobs/JREGIONAL_RUN_VX_GRIDSTAT | 40 +- jobs/JREGIONAL_RUN_VX_POINTSTAT | 40 +- scripts/exregional_get_ccpa_files.sh | 17 +- scripts/exregional_get_mrms_files.sh | 2 +- scripts/exregional_get_ndas_files.sh | 2 +- scripts/exregional_run_ensgridvx.sh | 180 ++++ scripts/exregional_run_ensgridvx_mean.sh | 189 +++++ scripts/exregional_run_ensgridvx_prob.sh | 189 +++++ scripts/exregional_run_enspointvx.sh | 170 ++++ scripts/exregional_run_enspointvx_mean.sh | 170 ++++ scripts/exregional_run_enspointvx_prob.sh | 170 ++++ scripts/exregional_run_gridstatvx.sh | 54 +- scripts/exregional_run_pointstatvx.sh | 36 +- tests/WE2E/run_WE2E_tests.sh | 5 +- .../config.MET_ensemble_verification.sh | 38 + ush/config.community.sh | 5 + ush/config_defaults.sh | 70 +- ush/generate_FV3LAM_wflow.sh | 58 ++ ush/setup.sh | 64 +- ush/templates/FV3LAM_wflow.xml | 778 ++++++++++++++++-- .../parm/met/EnsembleStatConfig_APCP | 257 ++++++ .../parm/met/EnsembleStatConfig_REFC | 265 ++++++ .../parm/met/EnsembleStatConfig_point | 290 +++++++ .../parm/met/GridStatConfig_APCP_mean | 178 ++++ .../parm/met/GridStatConfig_APCP_prob | 178 ++++ ...ridStatConfig_REFL => GridStatConfig_REFC} | 0 .../parm/met/GridStatConfig_REFC_mean | 219 +++++ .../parm/met/GridStatConfig_REFC_prob | 219 +++++ ush/templates/parm/met/PointStatConfig | 5 - ush/templates/parm/met/PointStatConfig_mean | 254 ++++++ ush/templates/parm/met/PointStatConfig_prob | 254 ++++++ ush/templates/parm/metplus/APCP_01h.conf | 18 +- ush/templates/parm/metplus/APCP_03h.conf | 23 +- ush/templates/parm/metplus/APCP_06h.conf | 22 +- ush/templates/parm/metplus/APCP_24h.conf | 22 +- .../parm/metplus/EnsembleStat_APCP01h.conf | 129 +++ .../parm/metplus/EnsembleStat_APCP03h.conf | 172 ++++ .../parm/metplus/EnsembleStat_APCP06h.conf | 172 ++++ .../parm/metplus/EnsembleStat_APCP24h.conf | 172 ++++ .../parm/metplus/EnsembleStat_REFC.conf | 119 +++ .../parm/metplus/EnsembleStat_RETOP.conf | 118 +++ .../parm/metplus/EnsembleStat_conus_sfc.conf | 170 ++++ .../parm/metplus/EnsembleStat_upper_air.conf | 173 ++++ .../parm/metplus/GridStat_APCP01h_mean.conf | 95 +++ .../parm/metplus/GridStat_APCP01h_prob.conf | 129 +++ .../parm/metplus/GridStat_APCP03h_mean.conf | 89 ++ .../parm/metplus/GridStat_APCP03h_prob.conf | 138 ++++ .../parm/metplus/GridStat_APCP06h_mean.conf | 89 ++ .../parm/metplus/GridStat_APCP06h_prob.conf | 138 ++++ .../parm/metplus/GridStat_APCP24h_mean.conf | 89 ++ .../parm/metplus/GridStat_APCP24h_prob.conf | 138 ++++ .../parm/metplus/GridStat_REFC_mean.conf | 94 +++ .../parm/metplus/GridStat_REFC_prob.conf | 122 +++ .../parm/metplus/GridStat_RETOP_mean.conf | 95 +++ .../parm/metplus/GridStat_RETOP_prob.conf | 122 +++ .../parm/metplus/PointStat_conus_sfc.conf | 67 +- .../metplus/PointStat_conus_sfc_mean.conf | 151 ++++ .../metplus/PointStat_conus_sfc_prob.conf | 195 +++++ .../parm/metplus/PointStat_upper_air.conf | 80 +- .../metplus/PointStat_upper_air_mean.conf | 152 ++++ .../metplus/PointStat_upper_air_prob.conf | 196 +++++ ush/templates/parm/metplus/REFC.conf | 20 +- ush/templates/parm/metplus/RETOP.conf | 20 +- ush/valid_param_vals.sh | 4 +- 70 files changed, 8162 insertions(+), 301 deletions(-) create mode 100755 jobs/JREGIONAL_RUN_VX_ENSGRID create mode 100755 jobs/JREGIONAL_RUN_VX_ENSGRID_MEAN create mode 100755 jobs/JREGIONAL_RUN_VX_ENSGRID_PROB create mode 100755 jobs/JREGIONAL_RUN_VX_ENSPOINT create mode 100755 jobs/JREGIONAL_RUN_VX_ENSPOINT_MEAN create mode 100755 jobs/JREGIONAL_RUN_VX_ENSPOINT_PROB create mode 100755 scripts/exregional_run_ensgridvx.sh create mode 100755 scripts/exregional_run_ensgridvx_mean.sh create mode 100755 scripts/exregional_run_ensgridvx_prob.sh create mode 100755 scripts/exregional_run_enspointvx.sh create mode 100755 scripts/exregional_run_enspointvx_mean.sh create mode 100755 scripts/exregional_run_enspointvx_prob.sh create mode 100644 tests/WE2E/test_configs/wflow_features/config.MET_ensemble_verification.sh create mode 100755 ush/templates/parm/met/EnsembleStatConfig_APCP create mode 100755 ush/templates/parm/met/EnsembleStatConfig_REFC create mode 100644 ush/templates/parm/met/EnsembleStatConfig_point create mode 100755 ush/templates/parm/met/GridStatConfig_APCP_mean create mode 100755 ush/templates/parm/met/GridStatConfig_APCP_prob rename ush/templates/parm/met/{GridStatConfig_REFL => GridStatConfig_REFC} (100%) create mode 100644 ush/templates/parm/met/GridStatConfig_REFC_mean create mode 100644 ush/templates/parm/met/GridStatConfig_REFC_prob create mode 100644 ush/templates/parm/met/PointStatConfig_mean create mode 100644 ush/templates/parm/met/PointStatConfig_prob create mode 100644 ush/templates/parm/metplus/EnsembleStat_APCP01h.conf create mode 100644 ush/templates/parm/metplus/EnsembleStat_APCP03h.conf create mode 100644 ush/templates/parm/metplus/EnsembleStat_APCP06h.conf create mode 100644 ush/templates/parm/metplus/EnsembleStat_APCP24h.conf create mode 100644 ush/templates/parm/metplus/EnsembleStat_REFC.conf create mode 100644 ush/templates/parm/metplus/EnsembleStat_RETOP.conf create mode 100644 ush/templates/parm/metplus/EnsembleStat_conus_sfc.conf create mode 100644 ush/templates/parm/metplus/EnsembleStat_upper_air.conf create mode 100644 ush/templates/parm/metplus/GridStat_APCP01h_mean.conf create mode 100644 ush/templates/parm/metplus/GridStat_APCP01h_prob.conf create mode 100644 ush/templates/parm/metplus/GridStat_APCP03h_mean.conf create mode 100644 ush/templates/parm/metplus/GridStat_APCP03h_prob.conf create mode 100644 ush/templates/parm/metplus/GridStat_APCP06h_mean.conf create mode 100644 ush/templates/parm/metplus/GridStat_APCP06h_prob.conf create mode 100644 ush/templates/parm/metplus/GridStat_APCP24h_mean.conf create mode 100644 ush/templates/parm/metplus/GridStat_APCP24h_prob.conf create mode 100644 ush/templates/parm/metplus/GridStat_REFC_mean.conf create mode 100644 ush/templates/parm/metplus/GridStat_REFC_prob.conf create mode 100644 ush/templates/parm/metplus/GridStat_RETOP_mean.conf create mode 100644 ush/templates/parm/metplus/GridStat_RETOP_prob.conf create mode 100644 ush/templates/parm/metplus/PointStat_conus_sfc_mean.conf create mode 100644 ush/templates/parm/metplus/PointStat_conus_sfc_prob.conf create mode 100644 ush/templates/parm/metplus/PointStat_upper_air_mean.conf create mode 100644 ush/templates/parm/metplus/PointStat_upper_air_prob.conf diff --git a/jobs/JREGIONAL_RUN_VX_ENSGRID b/jobs/JREGIONAL_RUN_VX_ENSGRID new file mode 100755 index 000000000..9fa88c2f7 --- /dev/null +++ b/jobs/JREGIONAL_RUN_VX_ENSGRID @@ -0,0 +1,90 @@ +#!/bin/bash + +# +#----------------------------------------------------------------------- +# +# This script runs METplus for ensemble-stat on the UPP output files by +# initialization time for all forecast hours for gridded analysis. +# +#----------------------------------------------------------------------- +# + +# +#----------------------------------------------------------------------- +# +# Source the variable definitions file and the bash utility functions. +# +#----------------------------------------------------------------------- +# +. ${GLOBAL_VAR_DEFNS_FP} +. $USHDIR/source_util_funcs.sh +# +#----------------------------------------------------------------------- +# +# Save current shell options (in a global array). Then set new options +# for this script/function. +# +#----------------------------------------------------------------------- +# +{ save_shell_opts; set -u -x; } > /dev/null 2>&1 +# +#----------------------------------------------------------------------- +# +# Get the full path to the file in which this script/function is located +# (scrfunc_fp), the name of that file (scrfunc_fn), and the directory in +# which the file is located (scrfunc_dir). +# +#----------------------------------------------------------------------- +# +scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +scrfunc_fn=$( basename "${scrfunc_fp}" ) +scrfunc_dir=$( dirname "${scrfunc_fp}" ) +# +#----------------------------------------------------------------------- +# +# Print message indicating entry into script. +# +#----------------------------------------------------------------------- +# +print_info_msg " +======================================================================== +Entering script: \"${scrfunc_fn}\" +In directory: \"${scrfunc_dir}\" + +This is the J-job script for the task that runs METplus for ensemble-stat +on gridded fields by initialization time for all forecast hours. +========================================================================" +# +#----------------------------------------------------------------------- +# +# Call the ex-script for this J-job and pass to it the necessary varia- +# bles. +# +#----------------------------------------------------------------------- +# +$SCRIPTSDIR/exregional_run_ensgridvx.sh \ + cycle_dir="${CYCLE_DIR}" || \ +print_err_msg_exit "\ +Call to ex-script corresponding to J-job \"${scrfunc_fn}\" failed." +# +#----------------------------------------------------------------------- +# +# Print exit message. +# +#----------------------------------------------------------------------- +# +print_info_msg " +======================================================================== +Exiting script: \"${scrfunc_fn}\" +In directory: \"${scrfunc_dir}\" +========================================================================" +# +#----------------------------------------------------------------------- +# +# Restore the shell options saved at the beginning of this script/func- +# tion. +# +#----------------------------------------------------------------------- +# +{ restore_shell_opts; } > /dev/null 2>&1 + diff --git a/jobs/JREGIONAL_RUN_VX_ENSGRID_MEAN b/jobs/JREGIONAL_RUN_VX_ENSGRID_MEAN new file mode 100755 index 000000000..67945ca7b --- /dev/null +++ b/jobs/JREGIONAL_RUN_VX_ENSGRID_MEAN @@ -0,0 +1,90 @@ +#!/bin/bash + +# +#----------------------------------------------------------------------- +# +# This script runs METplus for grid-stat on the UPP output files by +# initialization time for all forecast hours for gridded analysis. +# +#----------------------------------------------------------------------- +# + +# +#----------------------------------------------------------------------- +# +# Source the variable definitions file and the bash utility functions. +# +#----------------------------------------------------------------------- +# +. ${GLOBAL_VAR_DEFNS_FP} +. $USHDIR/source_util_funcs.sh +# +#----------------------------------------------------------------------- +# +# Save current shell options (in a global array). Then set new options +# for this script/function. +# +#----------------------------------------------------------------------- +# +{ save_shell_opts; set -u -x; } > /dev/null 2>&1 +# +#----------------------------------------------------------------------- +# +# Get the full path to the file in which this script/function is located +# (scrfunc_fp), the name of that file (scrfunc_fn), and the directory in +# which the file is located (scrfunc_dir). +# +#----------------------------------------------------------------------- +# +scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +scrfunc_fn=$( basename "${scrfunc_fp}" ) +scrfunc_dir=$( dirname "${scrfunc_fp}" ) +# +#----------------------------------------------------------------------- +# +# Print message indicating entry into script. +# +#----------------------------------------------------------------------- +# +print_info_msg " +======================================================================== +Entering script: \"${scrfunc_fn}\" +In directory: \"${scrfunc_dir}\" + +This is the J-job script for the task that runs METplus for grid-stat +on gridded fields by initialization time for all forecast hours. +========================================================================" +# +#----------------------------------------------------------------------- +# +# Call the ex-script for this J-job and pass to it the necessary varia- +# bles. +# +#----------------------------------------------------------------------- +# +$SCRIPTSDIR/exregional_run_ensgridvx_mean.sh \ + cycle_dir="${CYCLE_DIR}" || \ +print_err_msg_exit "\ +Call to ex-script corresponding to J-job \"${scrfunc_fn}\" failed." +# +#----------------------------------------------------------------------- +# +# Print exit message. +# +#----------------------------------------------------------------------- +# +print_info_msg " +======================================================================== +Exiting script: \"${scrfunc_fn}\" +In directory: \"${scrfunc_dir}\" +========================================================================" +# +#----------------------------------------------------------------------- +# +# Restore the shell options saved at the beginning of this script/func- +# tion. +# +#----------------------------------------------------------------------- +# +{ restore_shell_opts; } > /dev/null 2>&1 + diff --git a/jobs/JREGIONAL_RUN_VX_ENSGRID_PROB b/jobs/JREGIONAL_RUN_VX_ENSGRID_PROB new file mode 100755 index 000000000..f792d4b77 --- /dev/null +++ b/jobs/JREGIONAL_RUN_VX_ENSGRID_PROB @@ -0,0 +1,90 @@ +#!/bin/bash + +# +#----------------------------------------------------------------------- +# +# This script runs METplus for grid-stat on the UPP output files by +# initialization time for all forecast hours for gridded analysis. +# +#----------------------------------------------------------------------- +# + +# +#----------------------------------------------------------------------- +# +# Source the variable definitions file and the bash utility functions. +# +#----------------------------------------------------------------------- +# +. ${GLOBAL_VAR_DEFNS_FP} +. $USHDIR/source_util_funcs.sh +# +#----------------------------------------------------------------------- +# +# Save current shell options (in a global array). Then set new options +# for this script/function. +# +#----------------------------------------------------------------------- +# +{ save_shell_opts; set -u -x; } > /dev/null 2>&1 +# +#----------------------------------------------------------------------- +# +# Get the full path to the file in which this script/function is located +# (scrfunc_fp), the name of that file (scrfunc_fn), and the directory in +# which the file is located (scrfunc_dir). +# +#----------------------------------------------------------------------- +# +scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +scrfunc_fn=$( basename "${scrfunc_fp}" ) +scrfunc_dir=$( dirname "${scrfunc_fp}" ) +# +#----------------------------------------------------------------------- +# +# Print message indicating entry into script. +# +#----------------------------------------------------------------------- +# +print_info_msg " +======================================================================== +Entering script: \"${scrfunc_fn}\" +In directory: \"${scrfunc_dir}\" + +This is the J-job script for the task that runs METplus for grid-stat +on gridded fields by initialization time for all forecast hours. +========================================================================" +# +#----------------------------------------------------------------------- +# +# Call the ex-script for this J-job and pass to it the necessary varia- +# bles. +# +#----------------------------------------------------------------------- +# +$SCRIPTSDIR/exregional_run_ensgridvx_prob.sh \ + cycle_dir="${CYCLE_DIR}" || \ +print_err_msg_exit "\ +Call to ex-script corresponding to J-job \"${scrfunc_fn}\" failed." +# +#----------------------------------------------------------------------- +# +# Print exit message. +# +#----------------------------------------------------------------------- +# +print_info_msg " +======================================================================== +Exiting script: \"${scrfunc_fn}\" +In directory: \"${scrfunc_dir}\" +========================================================================" +# +#----------------------------------------------------------------------- +# +# Restore the shell options saved at the beginning of this script/func- +# tion. +# +#----------------------------------------------------------------------- +# +{ restore_shell_opts; } > /dev/null 2>&1 + diff --git a/jobs/JREGIONAL_RUN_VX_ENSPOINT b/jobs/JREGIONAL_RUN_VX_ENSPOINT new file mode 100755 index 000000000..dac8873a2 --- /dev/null +++ b/jobs/JREGIONAL_RUN_VX_ENSPOINT @@ -0,0 +1,88 @@ +#!/bin/bash + +# +#----------------------------------------------------------------------- +# +# +#----------------------------------------------------------------------- +# + +# +#----------------------------------------------------------------------- +# +# Source the variable definitions file and the bash utility functions. +# +#----------------------------------------------------------------------- +# +. ${GLOBAL_VAR_DEFNS_FP} +. $USHDIR/source_util_funcs.sh +# +#----------------------------------------------------------------------- +# +# Save current shell options (in a global array). Then set new options +# for this script/function. +# +#----------------------------------------------------------------------- +# +{ save_shell_opts; set -u -x; } > /dev/null 2>&1 +# +#----------------------------------------------------------------------- +# +# Get the full path to the file in which this script/function is located +# (scrfunc_fp), the name of that file (scrfunc_fn), and the directory in +# which the file is located (scrfunc_dir). +# +#----------------------------------------------------------------------- +# +scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +scrfunc_fn=$( basename "${scrfunc_fp}" ) +scrfunc_dir=$( dirname "${scrfunc_fp}" ) +# +#----------------------------------------------------------------------- +# +# Print message indicating entry into script. +# +#----------------------------------------------------------------------- +# +print_info_msg " +======================================================================== +Entering script: \"${scrfunc_fn}\" +In directory: \"${scrfunc_dir}\" + +This is the J-job script for the task that runs METplus for point-stat +by initialitation time for all forecast hours. +========================================================================" +# +#----------------------------------------------------------------------- +# +# Call the ex-script for this J-job and pass to it the necessary varia- +# bles. +# +#----------------------------------------------------------------------- +# +$SCRIPTSDIR/exregional_run_enspointvx.sh \ + cycle_dir="${CYCLE_DIR}" || \ +print_err_msg_exit "\ +Call to ex-script corresponding to J-job \"${scrfunc_fn}\" failed." +# +#----------------------------------------------------------------------- +# +# Print exit message. +# +#----------------------------------------------------------------------- +# +print_info_msg " +======================================================================== +Exiting script: \"${scrfunc_fn}\" +In directory: \"${scrfunc_dir}\" +========================================================================" +# +#----------------------------------------------------------------------- +# +# Restore the shell options saved at the beginning of this script/func- +# tion. +# +#----------------------------------------------------------------------- +# +{ restore_shell_opts; } > /dev/null 2>&1 + diff --git a/jobs/JREGIONAL_RUN_VX_ENSPOINT_MEAN b/jobs/JREGIONAL_RUN_VX_ENSPOINT_MEAN new file mode 100755 index 000000000..3ae1cb56d --- /dev/null +++ b/jobs/JREGIONAL_RUN_VX_ENSPOINT_MEAN @@ -0,0 +1,88 @@ +#!/bin/bash + +# +#----------------------------------------------------------------------- +# +# +#----------------------------------------------------------------------- +# + +# +#----------------------------------------------------------------------- +# +# Source the variable definitions file and the bash utility functions. +# +#----------------------------------------------------------------------- +# +. ${GLOBAL_VAR_DEFNS_FP} +. $USHDIR/source_util_funcs.sh +# +#----------------------------------------------------------------------- +# +# Save current shell options (in a global array). Then set new options +# for this script/function. +# +#----------------------------------------------------------------------- +# +{ save_shell_opts; set -u -x; } > /dev/null 2>&1 +# +#----------------------------------------------------------------------- +# +# Get the full path to the file in which this script/function is located +# (scrfunc_fp), the name of that file (scrfunc_fn), and the directory in +# which the file is located (scrfunc_dir). +# +#----------------------------------------------------------------------- +# +scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +scrfunc_fn=$( basename "${scrfunc_fp}" ) +scrfunc_dir=$( dirname "${scrfunc_fp}" ) +# +#----------------------------------------------------------------------- +# +# Print message indicating entry into script. +# +#----------------------------------------------------------------------- +# +print_info_msg " +======================================================================== +Entering script: \"${scrfunc_fn}\" +In directory: \"${scrfunc_dir}\" + +This is the J-job script for the task that runs METplus for point-stat +by initialitation time for all forecast hours. +========================================================================" +# +#----------------------------------------------------------------------- +# +# Call the ex-script for this J-job and pass to it the necessary varia- +# bles. +# +#----------------------------------------------------------------------- +# +$SCRIPTSDIR/exregional_run_enspointvx_mean.sh \ + cycle_dir="${CYCLE_DIR}" || \ +print_err_msg_exit "\ +Call to ex-script corresponding to J-job \"${scrfunc_fn}\" failed." +# +#----------------------------------------------------------------------- +# +# Print exit message. +# +#----------------------------------------------------------------------- +# +print_info_msg " +======================================================================== +Exiting script: \"${scrfunc_fn}\" +In directory: \"${scrfunc_dir}\" +========================================================================" +# +#----------------------------------------------------------------------- +# +# Restore the shell options saved at the beginning of this script/func- +# tion. +# +#----------------------------------------------------------------------- +# +{ restore_shell_opts; } > /dev/null 2>&1 + diff --git a/jobs/JREGIONAL_RUN_VX_ENSPOINT_PROB b/jobs/JREGIONAL_RUN_VX_ENSPOINT_PROB new file mode 100755 index 000000000..8810d0bcb --- /dev/null +++ b/jobs/JREGIONAL_RUN_VX_ENSPOINT_PROB @@ -0,0 +1,88 @@ +#!/bin/bash + +# +#----------------------------------------------------------------------- +# +# +#----------------------------------------------------------------------- +# + +# +#----------------------------------------------------------------------- +# +# Source the variable definitions file and the bash utility functions. +# +#----------------------------------------------------------------------- +# +. ${GLOBAL_VAR_DEFNS_FP} +. $USHDIR/source_util_funcs.sh +# +#----------------------------------------------------------------------- +# +# Save current shell options (in a global array). Then set new options +# for this script/function. +# +#----------------------------------------------------------------------- +# +{ save_shell_opts; set -u -x; } > /dev/null 2>&1 +# +#----------------------------------------------------------------------- +# +# Get the full path to the file in which this script/function is located +# (scrfunc_fp), the name of that file (scrfunc_fn), and the directory in +# which the file is located (scrfunc_dir). +# +#----------------------------------------------------------------------- +# +scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +scrfunc_fn=$( basename "${scrfunc_fp}" ) +scrfunc_dir=$( dirname "${scrfunc_fp}" ) +# +#----------------------------------------------------------------------- +# +# Print message indicating entry into script. +# +#----------------------------------------------------------------------- +# +print_info_msg " +======================================================================== +Entering script: \"${scrfunc_fn}\" +In directory: \"${scrfunc_dir}\" + +This is the J-job script for the task that runs METplus for point-stat +by initialitation time for all forecast hours. +========================================================================" +# +#----------------------------------------------------------------------- +# +# Call the ex-script for this J-job and pass to it the necessary varia- +# bles. +# +#----------------------------------------------------------------------- +# +$SCRIPTSDIR/exregional_run_enspointvx_prob.sh \ + cycle_dir="${CYCLE_DIR}" || \ +print_err_msg_exit "\ +Call to ex-script corresponding to J-job \"${scrfunc_fn}\" failed." +# +#----------------------------------------------------------------------- +# +# Print exit message. +# +#----------------------------------------------------------------------- +# +print_info_msg " +======================================================================== +Exiting script: \"${scrfunc_fn}\" +In directory: \"${scrfunc_dir}\" +========================================================================" +# +#----------------------------------------------------------------------- +# +# Restore the shell options saved at the beginning of this script/func- +# tion. +# +#----------------------------------------------------------------------- +# +{ restore_shell_opts; } > /dev/null 2>&1 + diff --git a/jobs/JREGIONAL_RUN_VX_GRIDSTAT b/jobs/JREGIONAL_RUN_VX_GRIDSTAT index ca29e2d93..300c79f9c 100755 --- a/jobs/JREGIONAL_RUN_VX_GRIDSTAT +++ b/jobs/JREGIONAL_RUN_VX_GRIDSTAT @@ -57,51 +57,13 @@ by initialization time for all forecast hours. # #----------------------------------------------------------------------- # -# If it doesn't already exist, create the directory (vx_dir) in -# which to store grid-stat output. -# -# Note that there may be a preexisting version of vx_dir from previous -# runs of this script (e.g. from the workflow task that runs this script -# failing and then being called again). Thus, we first make sure -# preexisting versions are deleted. -# -#----------------------------------------------------------------------- -# - -if [ "${RUN_ENVIR}" = "nco" ]; then - COMOUT="$COMROOT/$NET/$envir/$RUN.$PDY/$cyc" - postprd_dir="$COMOUT" -else - postprd_dir="${CYCLE_DIR}/postprd" -fi -mkdir_vrfy -p "${postprd_dir}" - -if [ "${RUN_ENVIR}" = "nco" ]; then - COMOUT="$COMROOT/$NET/$envir/$RUN.$PDY/$cyc" - vx_dir="$COMOUT" -else - vx_dir="${CYCLE_DIR}/metprd" -fi -mkdir_vrfy -p "${vx_dir}" - -gridstat_dir="${vx_dir}/grid_stat" -#check_for_preexist_dir "${gridstat_dir}" "delete" -mkdir_vrfy -p "${gridstat_dir}" - -cd_vrfy "${gridstat_dir}" -# -#----------------------------------------------------------------------- -# # Call the ex-script for this J-job and pass to it the necessary varia- # bles. # #----------------------------------------------------------------------- # $SCRIPTSDIR/exregional_run_gridstatvx.sh \ - cycle_dir="${CYCLE_DIR}" \ - postprd_dir="${postprd_dir}" \ - vx_dir="${vx_dir}" \ - gridstat_dir="${gridstat_dir}" || \ + cycle_dir="${CYCLE_DIR}" || \ print_err_msg_exit "\ Call to ex-script corresponding to J-job \"${scrfunc_fn}\" failed." # diff --git a/jobs/JREGIONAL_RUN_VX_POINTSTAT b/jobs/JREGIONAL_RUN_VX_POINTSTAT index f5c35253a..f67cb5547 100755 --- a/jobs/JREGIONAL_RUN_VX_POINTSTAT +++ b/jobs/JREGIONAL_RUN_VX_POINTSTAT @@ -55,51 +55,13 @@ by initialitation time for all forecast hours. # #----------------------------------------------------------------------- # -# If it doesn't already exist, create the directory (vx_dir) in -# which to store point-stat output. -# -# Note that there may be a preexisting version of vx_dir from previous -# runs of this script (e.g. from the workflow task that runs this script -# failing and then being called again). Thus, we first make sure -# preexisting versions are deleted. -# -#----------------------------------------------------------------------- -# - -if [ "${RUN_ENVIR}" = "nco" ]; then - COMOUT="$COMROOT/$NET/$envir/$RUN.$PDY/$cyc" - postprd_dir="$COMOUT" -else - postprd_dir="${CYCLE_DIR}/postprd" -fi -mkdir_vrfy -p "${postprd_dir}" - -if [ "${RUN_ENVIR}" = "nco" ]; then - COMOUT="$COMROOT/$NET/$envir/$RUN.$PDY/$cyc" - vx_dir="$COMOUT" -else - vx_dir="${CYCLE_DIR}/metprd" -fi -mkdir_vrfy -p "${vx_dir}" - -pointstat_dir="${vx_dir}/point_stat" -#check_for_preexist_dir "${pointstat_dir}" "delete" -mkdir_vrfy -p "${pointstat_dir}" - -cd_vrfy "${pointstat_dir}" -# -#----------------------------------------------------------------------- -# # Call the ex-script for this J-job and pass to it the necessary varia- # bles. # #----------------------------------------------------------------------- # $SCRIPTSDIR/exregional_run_pointstatvx.sh \ - cycle_dir="${CYCLE_DIR}" \ - postprd_dir="${postprd_dir}" \ - vx_dir="${vx_dir}" \ - pointstat_dir="${pointstat_dir}" || \ + cycle_dir="${CYCLE_DIR}" || \ print_err_msg_exit "\ Call to ex-script corresponding to J-job \"${scrfunc_fn}\" failed." # diff --git a/scripts/exregional_get_ccpa_files.sh b/scripts/exregional_get_ccpa_files.sh index 2354938f1..3c1504292 100755 --- a/scripts/exregional_get_ccpa_files.sh +++ b/scripts/exregional_get_ccpa_files.sh @@ -1,4 +1,4 @@ -#!/bin/sh +#!/bin/bash # This script reorganizes the CCPA data into a more intuitive structure: # A valid YYYYMMDD directory is created, and all files for the valid day are placed within the directory. @@ -150,7 +150,8 @@ while [[ ${current_fcst} -le ${fcst_length} ]]; do htar -xvf ${TarFile} `htar -tf ${TarFile} | egrep "ccpa.t${vhh}z.${accum}h.hrap.conus.gb2" | awk '{print $7}'` fi - # One hour CCPA files have incorrect metadeta in the files under the "00" directory. After data is pulled, reorganize into correct valid yyyymmdd structure. + # One hour CCPA files have incorrect metadeta in the files under the "00" directory from 20180718 to 20210504. + # After data is pulled, reorganize into correct valid yyyymmdd structure. if [[ ${vhh_noZero} -ge 1 && ${vhh_noZero} -le 6 ]]; then cp $ccpa_raw/${vyyyymmdd}/06/ccpa.t${vhh}z.${accum}h.hrap.conus.gb2 $ccpa_proc/${vyyyymmdd} elif [[ ${vhh_noZero} -ge 7 && ${vhh_noZero} -le 12 ]]; then @@ -158,9 +159,17 @@ while [[ ${current_fcst} -le ${fcst_length} ]]; do elif [[ ${vhh_noZero} -ge 13 && ${vhh_noZero} -le 18 ]]; then cp $ccpa_raw/${vyyyymmdd}/18/ccpa.t${vhh}z.${accum}h.hrap.conus.gb2 $ccpa_proc/${vyyyymmdd} elif [[ ${vhh_noZero} -ge 19 && ${vhh_noZero} -le 23 ]]; then - wgrib2 $ccpa_raw/${vyyyymmdd_p1}/00/ccpa.t${vhh}z.${accum}h.hrap.conus.gb2 -set_date -24hr -grib $ccpa_proc/${vyyyymmdd}/ccpa.t${vhh}z.${accum}h.hrap.conus.gb2 -s + if [[ ${vyyyymmdd} -ge 20180718 && ${vyyyymmdd} -le 20210504 ]]; then + wgrib2 $ccpa_raw/${vyyyymmdd_p1}/00/ccpa.t${vhh}z.${accum}h.hrap.conus.gb2 -set_date -24hr -grib $ccpa_proc/${vyyyymmdd}/ccpa.t${vhh}z.${accum}h.hrap.conus.gb2 -s + else + cp $ccpa_raw/${vyyyymmdd_p1}/00/ccpa.t${vhh}z.${accum}h.hrap.conus.gb2 $ccpa_proc/${vyyyymmdd} + fi elif [[ ${vhh_noZero} -eq 0 ]]; then - wgrib2 $ccpa_raw/${vyyyymmdd}/00/ccpa.t${vhh}z.${accum}h.hrap.conus.gb2 -set_date -24hr -grib $ccpa_proc/${vyyyymmdd}/ccpa.t${vhh}z.${accum}h.hrap.conus.gb2 -s + if [[ ${vyyyymmdd} -ge 20180718 && ${vyyyymmdd} -le 20210504 ]]; then + wgrib2 $ccpa_raw/${vyyyymmdd}/00/ccpa.t${vhh}z.${accum}h.hrap.conus.gb2 -set_date -24hr -grib $ccpa_proc/${vyyyymmdd}/ccpa.t${vhh}z.${accum}h.hrap.conus.gb2 -s + else + cp $ccpa_raw/${vyyyymmdd}/00/ccpa.t${vhh}z.${accum}h.hrap.conus.gb2 $ccpa_proc/${vyyyymmdd} + fi fi elif [[ ${accum} == "03" ]]; then diff --git a/scripts/exregional_get_mrms_files.sh b/scripts/exregional_get_mrms_files.sh index ff09c5546..8819959ec 100755 --- a/scripts/exregional_get_mrms_files.sh +++ b/scripts/exregional_get_mrms_files.sh @@ -1,4 +1,4 @@ -#!/bin/sh +#!/bin/bash . ${GLOBAL_VAR_DEFNS_FP} diff --git a/scripts/exregional_get_ndas_files.sh b/scripts/exregional_get_ndas_files.sh index f79acf77f..e1ec7f8d6 100755 --- a/scripts/exregional_get_ndas_files.sh +++ b/scripts/exregional_get_ndas_files.sh @@ -1,4 +1,4 @@ -#!/bin/sh +#!/bin/bash # This script reorganizes the NDAS data into a more intuitive structure: # A valid YYYYMMDD directory is created, and all files for the valid day are placed within the directory. diff --git a/scripts/exregional_run_ensgridvx.sh b/scripts/exregional_run_ensgridvx.sh new file mode 100755 index 000000000..ce0654753 --- /dev/null +++ b/scripts/exregional_run_ensgridvx.sh @@ -0,0 +1,180 @@ +#!/bin/bash +set -x + +# +#----------------------------------------------------------------------- +# +# Source the variable definitions file and the bash utility functions. +# +#----------------------------------------------------------------------- +# +. ${GLOBAL_VAR_DEFNS_FP} +. $USHDIR/source_util_funcs.sh +# +#----------------------------------------------------------------------- +# +# Save current shell options (in a global array). Then set new options +# for this script/function. +# +#----------------------------------------------------------------------- +# +{ save_shell_opts; set -u +x; } > /dev/null 2>&1 +# +#----------------------------------------------------------------------- +# +# Get the full path to the file in which this script/function is located +# (scrfunc_fp), the name of that file (scrfunc_fn), and the directory in +# which the file is located (scrfunc_dir). +# +#----------------------------------------------------------------------- +# +scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +scrfunc_fn=$( basename "${scrfunc_fp}" ) +scrfunc_dir=$( dirname "${scrfunc_fp}" ) +# +#----------------------------------------------------------------------- +# +# Print message indicating entry into script. +# +#----------------------------------------------------------------------- +# +print_info_msg " +======================================================================== +Entering script: \"${scrfunc_fn}\" +In directory: \"${scrfunc_dir}\" + +This is the ex-script for the task that runs METplus for ensemble-stat on +the UPP output files by initialization time for all forecast hours for +gridded data. +========================================================================" + +# +#----------------------------------------------------------------------- +# +# Specify the set of valid argument names for this script/function. +# Then process the arguments provided to this script/function (which +# should consist of a set of name-value pairs of the form arg1="value1", +# etc). +# +#----------------------------------------------------------------------- +# +valid_args=( "cycle_dir" ) +process_args valid_args "$@" +# +#----------------------------------------------------------------------- +# +# For debugging purposes, print out values of arguments passed to this +# script. Note that these will be printed out only if VERBOSE is set to +# TRUE. +# +#----------------------------------------------------------------------- +# +print_input_args valid_args + +#----------------------------------------------------------------------- +# +# Begin grid-to-grid ensemble vx. +# +#----------------------------------------------------------------------- +# +print_info_msg "$VERBOSE" "Starting ensemble-stat verification" + +# +#----------------------------------------------------------------------- +# +# Get the cycle date and hour (in formats of yyyymmdd and hh, respect- +# ively) from CDATE. Read in FHR and create a comma-separated list +# for METplus to run over. Determine the number padding needed based +# on number of ensemble members. +# +#----------------------------------------------------------------------- +# +yyyymmdd=${CDATE:0:8} +hh=${CDATE:8:2} +cyc=$hh +export CDATE +export hh + +fhr_last=`echo ${FHR} | awk '{ print $NF }'` +export fhr_last + +fhr_list=`echo ${FHR} | sed "s/ /,/g"` +export fhr_list + +NUM_PAD=${NDIGITS_ENSMEM_NAMES} + +# +#----------------------------------------------------------------------- +# +# Create LOG_SUFFIX to read into METplus conf files. +# +#----------------------------------------------------------------------- +# + +if [ ${VAR} == "APCP" ]; then + LOG_SUFFIX=ensgrid_${CDATE}_${VAR}_${ACCUM}h +else + LOG_SUFFIX=ensgrid_${CDATE}_${VAR} +fi + +# +#----------------------------------------------------------------------- +# +# Export some environment variables passed in by the XML +# +#----------------------------------------------------------------------- +# +export SCRIPTSDIR +export EXPTDIR +export MET_INSTALL_DIR +export MET_BIN_EXEC +export METPLUS_PATH +export METPLUS_CONF +export MET_CONFIG +export MODEL +export NET +export NUM_ENS_MEMBERS +export NUM_PAD +export LOG_SUFFIX + +# +#----------------------------------------------------------------------- +# +# Run METplus +# +#----------------------------------------------------------------------- +# +if [ ${VAR} == "APCP" ]; then + acc="${ACCUM}h" # for stats output prefix in EnsembleStatConfig + ${METPLUS_PATH}/ush/master_metplus.py \ + -c ${METPLUS_CONF}/common.conf \ + -c ${METPLUS_CONF}/EnsembleStat_${VAR}${acc}.conf +else + ${METPLUS_PATH}/ush/master_metplus.py \ + -c ${METPLUS_CONF}/common.conf \ + -c ${METPLUS_CONF}/EnsembleStat_${VAR}.conf +fi + +# +#----------------------------------------------------------------------- +# +# Print message indicating successful completion of script. +# +#----------------------------------------------------------------------- +# +print_info_msg " +======================================================================== +METplus ensemble-stat grid completed successfully. + +Exiting script: \"${scrfunc_fn}\" +In directory: \"${scrfunc_dir}\" +========================================================================" +# +#----------------------------------------------------------------------- +# +# Restore the shell options saved at the beginning of this script/func- +# tion. +# +#----------------------------------------------------------------------- +# +{ restore_shell_opts; } > /dev/null 2>&1 diff --git a/scripts/exregional_run_ensgridvx_mean.sh b/scripts/exregional_run_ensgridvx_mean.sh new file mode 100755 index 000000000..1a4d68aa8 --- /dev/null +++ b/scripts/exregional_run_ensgridvx_mean.sh @@ -0,0 +1,189 @@ +#!/bin/bash +set -x + +# +#----------------------------------------------------------------------- +# +# Source the variable definitions file and the bash utility functions. +# +#----------------------------------------------------------------------- +# +. ${GLOBAL_VAR_DEFNS_FP} +. $USHDIR/source_util_funcs.sh +# +#----------------------------------------------------------------------- +# +# Save current shell options (in a global array). Then set new options +# for this script/function. +# +#----------------------------------------------------------------------- +# +{ save_shell_opts; set -u +x; } > /dev/null 2>&1 +# +#----------------------------------------------------------------------- +# +# Get the full path to the file in which this script/function is located +# (scrfunc_fp), the name of that file (scrfunc_fn), and the directory in +# which the file is located (scrfunc_dir). +# +#----------------------------------------------------------------------- +# +scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +scrfunc_fn=$( basename "${scrfunc_fp}" ) +scrfunc_dir=$( dirname "${scrfunc_fp}" ) +# +#----------------------------------------------------------------------- +# +# Print message indicating entry into script. +# +#----------------------------------------------------------------------- +# +print_info_msg " +======================================================================== +Entering script: \"${scrfunc_fn}\" +In directory: \"${scrfunc_dir}\" + +This is the ex-script for the task that runs METplus for grid-stat on +the UPP output files by initialization time for all forecast hours for +gridded data. +========================================================================" + +# +#----------------------------------------------------------------------- +# +# Specify the set of valid argument names for this script/function. +# Then process the arguments provided to this script/function (which +# should consist of a set of name-value pairs of the form arg1="value1", +# etc). +# +#----------------------------------------------------------------------- +# +valid_args=( "cycle_dir" ) +process_args valid_args "$@" +# +#----------------------------------------------------------------------- +# +# For debugging purposes, print out values of arguments passed to this +# script. Note that these will be printed out only if VERBOSE is set to +# TRUE. +# +#----------------------------------------------------------------------- +# +print_input_args valid_args + +#----------------------------------------------------------------------- +# +# Begin grid-to-grid vx on ensemble output. +# +#----------------------------------------------------------------------- +# +print_info_msg "$VERBOSE" "Starting grid-stat verification" + +# +#----------------------------------------------------------------------- +# +# Get the cycle date and hour (in formats of yyyymmdd and hh, respect- +# ively) from CDATE. Also read in FHR and create a comma-separated list +# for METplus to run over. +# +#----------------------------------------------------------------------- +# +yyyymmdd=${CDATE:0:8} +hh=${CDATE:8:2} +cyc=$hh +export CDATE +export hh + +fhr_last=`echo ${FHR} | awk '{ print $NF }'` +export fhr_last + +fhr_list=`echo ${FHR} | sed "s/ /,/g"` +export fhr_list + +# +#----------------------------------------------------------------------- +# +# Create INPUT_BASE and LOG_SUFFIX to read into METplus conf files. +# +#----------------------------------------------------------------------- +# +INPUT_BASE=${EXPTDIR}/${CDATE}/metprd/ensemble_stat + +if [ ${VAR} == "APCP" ]; then + LOG_SUFFIX=ensgrid_mean_${CDATE}_${VAR}_${ACCUM}h +else + LOG_SUFFIX=ensgrid_mean_${CDATE}_${VAR} +fi + +# +#----------------------------------------------------------------------- +# +# Check for existence of top-level OBS_DIR +# +#----------------------------------------------------------------------- +# +if [[ ! -d "$OBS_DIR" ]]; then + print_err_msg_exit "\ + Exiting: OBS_DIR does not exist." +fi + +# +#----------------------------------------------------------------------- +# +# Export some environment variables passed in by the XML +# +#----------------------------------------------------------------------- +# +export SCRIPTSDIR +export EXPTDIR +export MET_INSTALL_DIR +export MET_BIN_EXEC +export METPLUS_PATH +export METPLUS_CONF +export MET_CONFIG +export MODEL +export NET +export INPUT_BASE +export LOG_SUFFIX + +# +#----------------------------------------------------------------------- +# +# Run METplus +# +#----------------------------------------------------------------------- +# +if [ ${VAR} == "APCP" ]; then + export acc="${ACCUM}h" + ${METPLUS_PATH}/ush/master_metplus.py \ + -c ${METPLUS_CONF}/common.conf \ + -c ${METPLUS_CONF}/GridStat_${VAR}${acc}_mean.conf +else + ${METPLUS_PATH}/ush/master_metplus.py \ + -c ${METPLUS_CONF}/common.conf \ + -c ${METPLUS_CONF}/GridStat_${VAR}_mean.conf +fi + +# +#----------------------------------------------------------------------- +# +# Print message indicating successful completion of script. +# +#----------------------------------------------------------------------- +# +print_info_msg " +======================================================================== +METplus grid-stat completed successfully. + +Exiting script: \"${scrfunc_fn}\" +In directory: \"${scrfunc_dir}\" +========================================================================" +# +#----------------------------------------------------------------------- +# +# Restore the shell options saved at the beginning of this script/func- +# tion. +# +#----------------------------------------------------------------------- +# +{ restore_shell_opts; } > /dev/null 2>&1 diff --git a/scripts/exregional_run_ensgridvx_prob.sh b/scripts/exregional_run_ensgridvx_prob.sh new file mode 100755 index 000000000..d0c948e33 --- /dev/null +++ b/scripts/exregional_run_ensgridvx_prob.sh @@ -0,0 +1,189 @@ +#!/bin/bash +set -x + +# +#----------------------------------------------------------------------- +# +# Source the variable definitions file and the bash utility functions. +# +#----------------------------------------------------------------------- +# +. ${GLOBAL_VAR_DEFNS_FP} +. $USHDIR/source_util_funcs.sh +# +#----------------------------------------------------------------------- +# +# Save current shell options (in a global array). Then set new options +# for this script/function. +# +#----------------------------------------------------------------------- +# +{ save_shell_opts; set -u +x; } > /dev/null 2>&1 +# +#----------------------------------------------------------------------- +# +# Get the full path to the file in which this script/function is located +# (scrfunc_fp), the name of that file (scrfunc_fn), and the directory in +# which the file is located (scrfunc_dir). +# +#----------------------------------------------------------------------- +# +scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +scrfunc_fn=$( basename "${scrfunc_fp}" ) +scrfunc_dir=$( dirname "${scrfunc_fp}" ) +# +#----------------------------------------------------------------------- +# +# Print message indicating entry into script. +# +#----------------------------------------------------------------------- +# +print_info_msg " +======================================================================== +Entering script: \"${scrfunc_fn}\" +In directory: \"${scrfunc_dir}\" + +This is the ex-script for the task that runs METplus for grid-stat on +the UPP output files by initialization time for all forecast hours for +gridded data. +========================================================================" + +# +#----------------------------------------------------------------------- +# +# Specify the set of valid argument names for this script/function. +# Then process the arguments provided to this script/function (which +# should consist of a set of name-value pairs of the form arg1="value1", +# etc). +# +#----------------------------------------------------------------------- +# +valid_args=( "cycle_dir" ) +process_args valid_args "$@" +# +#----------------------------------------------------------------------- +# +# For debugging purposes, print out values of arguments passed to this +# script. Note that these will be printed out only if VERBOSE is set to +# TRUE. +# +#----------------------------------------------------------------------- +# +print_input_args valid_args + +#----------------------------------------------------------------------- +# +# Begin grid-to-grid vx on ensemble output. +# +#----------------------------------------------------------------------- +# +print_info_msg "$VERBOSE" "Starting grid-stat verification" + +# +#----------------------------------------------------------------------- +# +# Get the cycle date and hour (in formats of yyyymmdd and hh, respect- +# ively) from CDATE. Also read in FHR and create a comma-separated list +# for METplus to run over. +# +#----------------------------------------------------------------------- +# +yyyymmdd=${CDATE:0:8} +hh=${CDATE:8:2} +cyc=$hh +export CDATE +export hh + +fhr_last=`echo ${FHR} | awk '{ print $NF }'` +export fhr_last + +fhr_list=`echo ${FHR} | sed "s/ /,/g"` +export fhr_list + +# +#----------------------------------------------------------------------- +# +# Create INPUT_BASE and LOG_SUFFIX to read into METplus conf files. +# +#----------------------------------------------------------------------- +# +INPUT_BASE=${EXPTDIR}/${CDATE}/metprd/ensemble_stat + +if [ ${VAR} == "APCP" ]; then + LOG_SUFFIX=ensgrid_prob_${CDATE}_${VAR}_${ACCUM}h +else + LOG_SUFFIX=ensgrid_prob_${CDATE}_${VAR} +fi + +# +#----------------------------------------------------------------------- +# +# Check for existence of top-level OBS_DIR +# +#----------------------------------------------------------------------- +# +if [[ ! -d "$OBS_DIR" ]]; then + print_err_msg_exit "\ + Exiting: OBS_DIR does not exist." +fi + +# +#----------------------------------------------------------------------- +# +# Export some environment variables passed in by the XML +# +#----------------------------------------------------------------------- +# +export SCRIPTSDIR +export INPUT_BASE +export EXPTDIR +export MET_INSTALL_DIR +export MET_BIN_EXEC +export METPLUS_PATH +export METPLUS_CONF +export MET_CONFIG +export MODEL +export NET +export LOG_SUFFIX + +# +#----------------------------------------------------------------------- +# +# Run METplus +# +#----------------------------------------------------------------------- +# +if [ ${VAR} == "APCP" ]; then + export acc="${ACCUM}h" + ${METPLUS_PATH}/ush/master_metplus.py \ + -c ${METPLUS_CONF}/common.conf \ + -c ${METPLUS_CONF}/GridStat_${VAR}${acc}_prob.conf +else + ${METPLUS_PATH}/ush/master_metplus.py \ + -c ${METPLUS_CONF}/common.conf \ + -c ${METPLUS_CONF}/GridStat_${VAR}_prob.conf +fi + +# +#----------------------------------------------------------------------- +# +# Print message indicating successful completion of script. +# +#----------------------------------------------------------------------- +# +print_info_msg " +======================================================================== +METplus grid-stat completed successfully. + +Exiting script: \"${scrfunc_fn}\" +In directory: \"${scrfunc_dir}\" +========================================================================" +# +#----------------------------------------------------------------------- +# +# Restore the shell options saved at the beginning of this script/func- +# tion. +# +#----------------------------------------------------------------------- +# +{ restore_shell_opts; } > /dev/null 2>&1 diff --git a/scripts/exregional_run_enspointvx.sh b/scripts/exregional_run_enspointvx.sh new file mode 100755 index 000000000..67f0017a3 --- /dev/null +++ b/scripts/exregional_run_enspointvx.sh @@ -0,0 +1,170 @@ +#!/bin/bash +set -x + +# +#----------------------------------------------------------------------- +# +# Source the variable definitions file and the bash utility functions. +# +#----------------------------------------------------------------------- +# +. ${GLOBAL_VAR_DEFNS_FP} +. $USHDIR/source_util_funcs.sh +# +#----------------------------------------------------------------------- +# +# Save current shell options (in a global array). Then set new options +# for this script/function. +# +#----------------------------------------------------------------------- +# +{ save_shell_opts; set -u +x; } > /dev/null 2>&1 +# +#----------------------------------------------------------------------- +# +# Get the full path to the file in which this script/function is located +# (scrfunc_fp), the name of that file (scrfunc_fn), and the directory in +# which the file is located (scrfunc_dir). +# +#----------------------------------------------------------------------- +# +scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +scrfunc_fn=$( basename "${scrfunc_fp}" ) +scrfunc_dir=$( dirname "${scrfunc_fp}" ) +# +#----------------------------------------------------------------------- +# +# Print message indicating entry into script. +# +#----------------------------------------------------------------------- +# +print_info_msg " +======================================================================== +Entering script: \"${scrfunc_fn}\" +In directory: \"${scrfunc_dir}\" + +This is the ex-script for the task that runs METplus for point-stat on +the UPP output files by initialization time for all forecast hours. +========================================================================" + +# +#----------------------------------------------------------------------- +# +# Specify the set of valid argument names for this script/function. +# Then process the arguments provided to this script/function (which +# should consist of a set of name-value pairs of the form arg1="value1", +# etc). +# +#----------------------------------------------------------------------- +# +valid_args=( "cycle_dir" ) +process_args valid_args "$@" +# +#----------------------------------------------------------------------- +# +# For debugging purposes, print out values of arguments passed to this +# script. Note that these will be printed out only if VERBOSE is set to +# TRUE. +# +#----------------------------------------------------------------------- +# +print_input_args valid_args +#----------------------------------------------------------------------- +# +# Begin grid-to-point ensemble vx. +# +#----------------------------------------------------------------------- +# +print_info_msg "$VERBOSE" "Starting point-based ensemble-stat verification" + +# +#----------------------------------------------------------------------- +# +# Get the cycle date and hour (in formats of yyyymmdd and hh, respect- +# ively) from CDATE. Also read in FHR and create a comma-separated list +# for METplus to run over. +# +#----------------------------------------------------------------------- +# +yyyymmdd=${CDATE:0:8} +hh=${CDATE:8:2} +cyc=$hh +export CDATE +export hh + +fhr_last=`echo ${FHR} | awk '{ print $NF }'` +export fhr_last + +fhr_list=`echo ${FHR} | sed "s/ /,/g"` +export fhr_list + +# +#----------------------------------------------------------------------- +# +# Create LOG_SUFFIX to read into METplus conf files. +# +#----------------------------------------------------------------------- +# +LOG_SUFFIX=enspoint_${CDATE} + +# +#----------------------------------------------------------------------- +# +# Check for existence of top-level OBS_DIR +# +#----------------------------------------------------------------------- +# +if [[ ! -d "$OBS_DIR" ]]; then + print_err_msg_exit "\ + Exiting: OBS_DIR does not exist." +fi + +# +#----------------------------------------------------------------------- +# +# Export some environment variables passed in by the XML and run METplus +# +#----------------------------------------------------------------------- +# +export EXPTDIR +export LOG_SUFFIX +export MET_INSTALL_DIR +export MET_BIN_EXEC +export METPLUS_PATH +export METPLUS_CONF +export MET_CONFIG +export MODEL +export NET +export NUM_ENS_MEMBERS + +${METPLUS_PATH}/ush/master_metplus.py \ + -c ${METPLUS_CONF}/common.conf \ + -c ${METPLUS_CONF}/EnsembleStat_conus_sfc.conf + +${METPLUS_PATH}/ush/master_metplus.py \ + -c ${METPLUS_CONF}/common.conf \ + -c ${METPLUS_CONF}/EnsembleStat_upper_air.conf + +# +#----------------------------------------------------------------------- +# +# Print message indicating successful completion of script. +# +#----------------------------------------------------------------------- +# +print_info_msg " +======================================================================== +METplus ensemble-stat completed successfully. + +Exiting script: \"${scrfunc_fn}\" +In directory: \"${scrfunc_dir}\" +========================================================================" +# +#----------------------------------------------------------------------- +# +# Restore the shell options saved at the beginning of this script/func- +# tion. +# +#----------------------------------------------------------------------- +# +{ restore_shell_opts; } > /dev/null 2>&1 diff --git a/scripts/exregional_run_enspointvx_mean.sh b/scripts/exregional_run_enspointvx_mean.sh new file mode 100755 index 000000000..ad673428a --- /dev/null +++ b/scripts/exregional_run_enspointvx_mean.sh @@ -0,0 +1,170 @@ +#!/bin/bash +set -x + +# +#----------------------------------------------------------------------- +# +# Source the variable definitions file and the bash utility functions. +# +#----------------------------------------------------------------------- +# +. ${GLOBAL_VAR_DEFNS_FP} +. $USHDIR/source_util_funcs.sh +# +#----------------------------------------------------------------------- +# +# Save current shell options (in a global array). Then set new options +# for this script/function. +# +#----------------------------------------------------------------------- +# +{ save_shell_opts; set -u +x; } > /dev/null 2>&1 +# +#----------------------------------------------------------------------- +# +# Get the full path to the file in which this script/function is located +# (scrfunc_fp), the name of that file (scrfunc_fn), and the directory in +# which the file is located (scrfunc_dir). +# +#----------------------------------------------------------------------- +# +scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +scrfunc_fn=$( basename "${scrfunc_fp}" ) +scrfunc_dir=$( dirname "${scrfunc_fp}" ) +# +#----------------------------------------------------------------------- +# +# Print message indicating entry into script. +# +#----------------------------------------------------------------------- +# +print_info_msg " +======================================================================== +Entering script: \"${scrfunc_fn}\" +In directory: \"${scrfunc_dir}\" + +This is the ex-script for the task that runs METplus for point-stat on +the UPP output files by initialization time for all forecast hours. +========================================================================" + +# +#----------------------------------------------------------------------- +# +# Specify the set of valid argument names for this script/function. +# Then process the arguments provided to this script/function (which +# should consist of a set of name-value pairs of the form arg1="value1", +# etc). +# +#----------------------------------------------------------------------- +# +valid_args=( "cycle_dir" ) +process_args valid_args "$@" +# +#----------------------------------------------------------------------- +# +# For debugging purposes, print out values of arguments passed to this +# script. Note that these will be printed out only if VERBOSE is set to +# TRUE. +# +#----------------------------------------------------------------------- +# +print_input_args valid_args +#----------------------------------------------------------------------- +# +# Begin grid-to-point vx on ensemble output. +# +#----------------------------------------------------------------------- +# +print_info_msg "$VERBOSE" "Starting point-stat verification" + +# +#----------------------------------------------------------------------- +# +# Get the cycle date and hour (in formats of yyyymmdd and hh, respect- +# ively) from CDATE. Also read in FHR and create a comma-separated list +# for METplus to run over. +# +#----------------------------------------------------------------------- +# +yyyymmdd=${CDATE:0:8} +hh=${CDATE:8:2} +cyc=$hh +export CDATE +export hh + +fhr_last=`echo ${FHR} | awk '{ print $NF }'` +export fhr_last + +fhr_list=`echo ${FHR} | sed "s/ /,/g"` +export fhr_list + +# +#----------------------------------------------------------------------- +# +# Create INPUT_BASE and LOG_SUFFIX to read into METplus conf files. +# +#----------------------------------------------------------------------- +# +INPUT_BASE=${EXPTDIR}/${CDATE}/metprd/ensemble_stat +LOG_SUFFIX=enspoint_mean_${CDATE} + +# +#----------------------------------------------------------------------- +# +# Check for existence of top-level OBS_DIR +# +#----------------------------------------------------------------------- +# +if [[ ! -d "$OBS_DIR" ]]; then + print_err_msg_exit "\ + Exiting: OBS_DIR does not exist." +fi + +# +#----------------------------------------------------------------------- +# +# Export some environment variables passed in by the XML and run METplus +# +#----------------------------------------------------------------------- +# +export EXPTDIR +export INPUT_BASE +export LOG_SUFFIX +export MET_INSTALL_DIR +export MET_BIN_EXEC +export METPLUS_PATH +export METPLUS_CONF +export MET_CONFIG +export MODEL +export NET + +${METPLUS_PATH}/ush/master_metplus.py \ + -c ${METPLUS_CONF}/common.conf \ + -c ${METPLUS_CONF}/PointStat_conus_sfc_mean.conf + +${METPLUS_PATH}/ush/master_metplus.py \ + -c ${METPLUS_CONF}/common.conf \ + -c ${METPLUS_CONF}/PointStat_upper_air_mean.conf +# +#----------------------------------------------------------------------- +# +# Print message indicating successful completion of script. +# +#----------------------------------------------------------------------- +# +print_info_msg " +======================================================================== +METplus ensemble-stat completed successfully. + +Exiting script: \"${scrfunc_fn}\" +In directory: \"${scrfunc_dir}\" +========================================================================" +# +#----------------------------------------------------------------------- +# +# Restore the shell options saved at the beginning of this script/func- +# tion. +# +#----------------------------------------------------------------------- +# +{ restore_shell_opts; } > /dev/null 2>&1 diff --git a/scripts/exregional_run_enspointvx_prob.sh b/scripts/exregional_run_enspointvx_prob.sh new file mode 100755 index 000000000..afecf2ff1 --- /dev/null +++ b/scripts/exregional_run_enspointvx_prob.sh @@ -0,0 +1,170 @@ +#!/bin/bash +set -x + +# +#----------------------------------------------------------------------- +# +# Source the variable definitions file and the bash utility functions. +# +#----------------------------------------------------------------------- +# +. ${GLOBAL_VAR_DEFNS_FP} +. $USHDIR/source_util_funcs.sh +# +#----------------------------------------------------------------------- +# +# Save current shell options (in a global array). Then set new options +# for this script/function. +# +#----------------------------------------------------------------------- +# +{ save_shell_opts; set -u +x; } > /dev/null 2>&1 +# +#----------------------------------------------------------------------- +# +# Get the full path to the file in which this script/function is located +# (scrfunc_fp), the name of that file (scrfunc_fn), and the directory in +# which the file is located (scrfunc_dir). +# +#----------------------------------------------------------------------- +# +scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +scrfunc_fn=$( basename "${scrfunc_fp}" ) +scrfunc_dir=$( dirname "${scrfunc_fp}" ) +# +#----------------------------------------------------------------------- +# +# Print message indicating entry into script. +# +#----------------------------------------------------------------------- +# +print_info_msg " +======================================================================== +Entering script: \"${scrfunc_fn}\" +In directory: \"${scrfunc_dir}\" + +This is the ex-script for the task that runs METplus for point-stat on +the UPP output files by initialization time for all forecast hours. +========================================================================" + +# +#----------------------------------------------------------------------- +# +# Specify the set of valid argument names for this script/function. +# Then process the arguments provided to this script/function (which +# should consist of a set of name-value pairs of the form arg1="value1", +# etc). +# +#----------------------------------------------------------------------- +# +valid_args=( "cycle_dir" ) +process_args valid_args "$@" +# +#----------------------------------------------------------------------- +# +# For debugging purposes, print out values of arguments passed to this +# script. Note that these will be printed out only if VERBOSE is set to +# TRUE. +# +#----------------------------------------------------------------------- +# +print_input_args valid_args +#----------------------------------------------------------------------- +# +# Begin grid-to-point vx on ensemble output. +# +#----------------------------------------------------------------------- +# +print_info_msg "$VERBOSE" "Starting point-stat verification" + +# +#----------------------------------------------------------------------- +# +# Get the cycle date and hour (in formats of yyyymmdd and hh, respect- +# ively) from CDATE. Also read in FHR and create a comma-separated list +# for METplus to run over. +# +#----------------------------------------------------------------------- +# +yyyymmdd=${CDATE:0:8} +hh=${CDATE:8:2} +cyc=$hh +export CDATE +export hh + +fhr_last=`echo ${FHR} | awk '{ print $NF }'` +export fhr_last + +fhr_list=`echo ${FHR} | sed "s/ /,/g"` +export fhr_list + +# +#----------------------------------------------------------------------- +# +# Create INPUT_BASE and LOG_SUFFIX to read into METplus conf files. +# +#----------------------------------------------------------------------- +# +INPUT_BASE=${EXPTDIR}/${CDATE}/metprd/ensemble_stat +LOG_SUFFIX=enspoint_prob_${CDATE} + +# +#----------------------------------------------------------------------- +# +# Check for existence of top-level OBS_DIR +# +#----------------------------------------------------------------------- +# +if [[ ! -d "$OBS_DIR" ]]; then + print_err_msg_exit "\ + Exiting: OBS_DIR does not exist." +fi + +# +#----------------------------------------------------------------------- +# +# Export some environment variables passed in by the XML and run METplus +# +#----------------------------------------------------------------------- +# +export EXPTDIR +export INPUT_BASE +export LOG_SUFFIX +export MET_INSTALL_DIR +export MET_BIN_EXEC +export METPLUS_PATH +export METPLUS_CONF +export MET_CONFIG +export MODEL +export NET + +${METPLUS_PATH}/ush/master_metplus.py \ + -c ${METPLUS_CONF}/common.conf \ + -c ${METPLUS_CONF}/PointStat_conus_sfc_prob.conf + +${METPLUS_PATH}/ush/master_metplus.py \ + -c ${METPLUS_CONF}/common.conf \ + -c ${METPLUS_CONF}/PointStat_upper_air_prob.conf +# +#----------------------------------------------------------------------- +# +# Print message indicating successful completion of script. +# +#----------------------------------------------------------------------- +# +print_info_msg " +======================================================================== +METplus ensemble-stat completed successfully. + +Exiting script: \"${scrfunc_fn}\" +In directory: \"${scrfunc_dir}\" +========================================================================" +# +#----------------------------------------------------------------------- +# +# Restore the shell options saved at the beginning of this script/func- +# tion. +# +#----------------------------------------------------------------------- +# +{ restore_shell_opts; } > /dev/null 2>&1 diff --git a/scripts/exregional_run_gridstatvx.sh b/scripts/exregional_run_gridstatvx.sh index 6b377f8c4..16b641f33 100755 --- a/scripts/exregional_run_gridstatvx.sh +++ b/scripts/exregional_run_gridstatvx.sh @@ -1,4 +1,4 @@ -#!/bin/sh -l +#!/bin/bash set -x # @@ -57,7 +57,7 @@ the UPP output files by initialization time for all forecast hours. # #----------------------------------------------------------------------- # -valid_args=( "cycle_dir" "postprd_dir" "vx_dir" "gridstat_dir" ) +valid_args=( "cycle_dir" ) process_args valid_args "$@" # #----------------------------------------------------------------------- @@ -70,16 +70,6 @@ process_args valid_args "$@" # print_input_args valid_args -#----------------------------------------------------------------------- -# -# Remove any files from previous runs and stage necessary files in gridstat_dir. -# -#----------------------------------------------------------------------- -# -print_info_msg "$VERBOSE" "Starting grid-stat verification" - -cd ${gridstat_dir} - # #----------------------------------------------------------------------- # @@ -101,6 +91,33 @@ export fhr_last fhr_list=`echo ${FHR} | sed "s/ /,/g"` export fhr_list +# +#----------------------------------------------------------------------- +# +# Create INPUT_BASE and LOG_SUFFIX to read into METplus conf files. +# +#----------------------------------------------------------------------- +# +if [[ ${DO_ENSEMBLE} == "FALSE" ]]; then + INPUT_BASE=${EXPTDIR}/${CDATE}/postprd + OUTPUT_BASE=${EXPTDIR}/${CDATE} + if [ ${VAR} == "APCP" ]; then + LOG_SUFFIX=gridstat_${CDATE}_${VAR}_${ACCUM}h + else + LOG_SUFFIX=gridstat_${CDATE}_${VAR} + fi +elif [[ ${DO_ENSEMBLE} == "TRUE" ]]; then + INPUT_BASE=${EXPTDIR}/${CDATE}/${SLASH_ENSMEM_SUBDIR}/postprd + OUTPUT_BASE=${EXPTDIR}/${CDATE}/${SLASH_ENSMEM_SUBDIR} + ENSMEM=`echo ${SLASH_ENSMEM_SUBDIR} | cut -d"/" -f2` + MODEL=${MODEL}_${ENSMEM} + if [ ${VAR} == "APCP" ]; then + LOG_SUFFIX=gridstat_${CDATE}_${ENSMEM}_${VAR}_${ACCUM}h + else + LOG_SUFFIX=gridstat_${CDATE}_${ENSMEM}_${VAR} + fi +fi + # #----------------------------------------------------------------------- # @@ -123,13 +140,14 @@ fi # export SCRIPTSDIR export EXPTDIR +export INPUT_BASE +export OUTPUT_BASE +export LOG_SUFFIX export MET_INSTALL_DIR export MET_BIN_EXEC export METPLUS_PATH export METPLUS_CONF export MET_CONFIG -export OBS_DIR -export VAR export MODEL export NET @@ -145,16 +163,10 @@ if [ ${VAR} == "APCP" ]; then ${METPLUS_PATH}/ush/master_metplus.py \ -c ${METPLUS_CONF}/common.conf \ -c ${METPLUS_CONF}/${VAR}_${acc}.conf -elif [ ${VAR} == "REFC" ]; then - ${METPLUS_PATH}/ush/master_metplus.py \ - -c ${METPLUS_CONF}/common.conf \ - -c ${METPLUS_CONF}/${VAR}.conf -elif [ ${VAR} == "RETOP" ]; then +else ${METPLUS_PATH}/ush/master_metplus.py \ -c ${METPLUS_CONF}/common.conf \ -c ${METPLUS_CONF}/${VAR}.conf -else - echo "No variable defined" fi # diff --git a/scripts/exregional_run_pointstatvx.sh b/scripts/exregional_run_pointstatvx.sh index 73572d9a6..fd68dfca8 100755 --- a/scripts/exregional_run_pointstatvx.sh +++ b/scripts/exregional_run_pointstatvx.sh @@ -1,4 +1,4 @@ -#!/bin/sh -l +#!/bin/bash set -x # @@ -57,7 +57,7 @@ the UPP output files by initialization time for all forecast hours. # #----------------------------------------------------------------------- # -valid_args=( "cycle_dir" "postprd_dir" "vx_dir" "pointstat_dir" ) +valid_args=( "cycle_dir" ) process_args valid_args "$@" # #----------------------------------------------------------------------- @@ -69,15 +69,6 @@ process_args valid_args "$@" #----------------------------------------------------------------------- # print_input_args valid_args -#----------------------------------------------------------------------- -# -# Remove any files from previous runs and stage necessary files in pointstat_dir. -# -#----------------------------------------------------------------------- -# -print_info_msg "$VERBOSE" "Starting point-stat verification" - -cd ${pointstat_dir} # #----------------------------------------------------------------------- @@ -100,6 +91,25 @@ export fhr_last fhr_list=`echo ${FHR} | sed "s/ /,/g"` export fhr_list +# +#----------------------------------------------------------------------- +# +# Create INPUT_BASE to read into METplus conf files. +# +#----------------------------------------------------------------------- +# +if [[ ${DO_ENSEMBLE} == "FALSE" ]]; then + INPUT_BASE=${EXPTDIR}/${CDATE}/postprd + OUTPUT_BASE=${EXPTDIR}/${CDATE} + LOG_SUFFIX=pointstat_${CDATE} +elif [[ ${DO_ENSEMBLE} == "TRUE" ]]; then + INPUT_BASE=${EXPTDIR}/${CDATE}/${SLASH_ENSMEM_SUBDIR}/postprd + OUTPUT_BASE=${EXPTDIR}/${CDATE}/${SLASH_ENSMEM_SUBDIR} + ENSMEM=`echo ${SLASH_ENSMEM_SUBDIR} | cut -d"/" -f2` + MODEL=${MODEL}_${ENSMEM} + LOG_SUFFIX=pointstat_${CDATE}_${ENSMEM} +fi + # #----------------------------------------------------------------------- # @@ -120,12 +130,14 @@ fi #----------------------------------------------------------------------- # export EXPTDIR +export INPUT_BASE +export OUTPUT_BASE +export LOG_SUFFIX export MET_INSTALL_DIR export MET_BIN_EXEC export METPLUS_PATH export METPLUS_CONF export MET_CONFIG -export OBS_DIR export MODEL export NET diff --git a/tests/WE2E/run_WE2E_tests.sh b/tests/WE2E/run_WE2E_tests.sh index 3ca9a2c01..b8eaeec74 100755 --- a/tests/WE2E/run_WE2E_tests.sh +++ b/tests/WE2E/run_WE2E_tests.sh @@ -950,7 +950,6 @@ external model files should be located has not been specified for this machine (MACHINE): MACHINE= \"${MACHINE}\"" fi - EXTRN_MDL_SOURCE_BASEDIR_ICS="${extrn_mdl_source_basedir}/${EXTRN_MDL_NAME_ICS}" if [ "${EXTRN_MDL_NAME_ICS}" = "FV3GFS" ] && [ "$MACHINE" = "HERA" ]; then EXTRN_MDL_SOURCE_BASEDIR_ICS="${EXTRN_MDL_SOURCE_BASEDIR_ICS}/${FV3GFS_FILE_FMT_ICS}" @@ -1023,7 +1022,9 @@ EXTRN_MDL_FILES_LBCS=( $( printf "\"%s\" " "${EXTRN_MDL_FILES_LBCS[@]}" ))" #----------------------------------------------------------------------- # if [ "${RUN_TASK_VX_GRIDSTAT}" = "TRUE" ] || \ - [ "${RUN_TASK_VX_POINTSTAT}" = "TRUE" ]; then + [ "${RUN_TASK_VX_POINTSTAT}" = "TRUE" ] || \ + [ "${RUN_TASK_VX_ENSGRID}" = "TRUE" ] || \ + [ "${RUN_TASK_VX_ENSPOINT}" = "TRUE" ]; then if [ "$MACHINE" = "WCOSS_CRAY" ]; then met_install_dir="/gpfs/hps3/emc/meso/noscrub/emc.metplus/met/10.0.0" diff --git a/tests/WE2E/test_configs/wflow_features/config.MET_ensemble_verification.sh b/tests/WE2E/test_configs/wflow_features/config.MET_ensemble_verification.sh new file mode 100644 index 000000000..078ce144f --- /dev/null +++ b/tests/WE2E/test_configs/wflow_features/config.MET_ensemble_verification.sh @@ -0,0 +1,38 @@ +# +# TEST PURPOSE/DESCRIPTION: +# ------------------------ +# +# This test is to ensure that the workflow running in community mode +# completes successfully with MET verification. +# + +RUN_ENVIR="community" +PREEXISTING_DIR_METHOD="rename" + +PREDEF_GRID_NAME="RRFS_CONUS_25km" +CCPP_PHYS_SUITE="FV3_GFS_v15p2" + +DO_ENSEMBLE="TRUE" +NUM_ENS_MEMBERS="2" + +USE_USER_STAGED_EXTRN_FILES="TRUE" + +EXTRN_MDL_NAME_ICS="FV3GFS" +FV3GFS_FILE_FMT_ICS="grib2" +EXTRN_MDL_NAME_LBCS="FV3GFS" +FV3GFS_FILE_FMT_LBCS="grib2" + +DATE_FIRST_CYCL="20190615" +DATE_LAST_CYCL="20190615" +CYCL_HRS=( "00" ) + +FCST_LEN_HRS="6" +LBC_SPEC_INTVL_HRS="6" + +MODEL="FV3_GFS_v15p2_CONUS_25km" +RUN_TASK_VX_GRIDSTAT="TRUE" +RUN_TASK_VX_POINTSTAT="TRUE" +RUN_TASK_VX_ENSGRID="TRUE" +RUN_TASK_VX_ENSPOINT="TRUE" + +WTIME_RUN_FCST="01:00:00" diff --git a/ush/config.community.sh b/ush/config.community.sh index b0c1f21f1..70433d597 100644 --- a/ush/config.community.sh +++ b/ush/config.community.sh @@ -11,6 +11,9 @@ PREEXISTING_DIR_METHOD="rename" PREDEF_GRID_NAME="RRFS_CONUS_25km" QUILTING="TRUE" +DO_ENSEMBLE="FALSE" +NUM_ENS_MEMBERS="2" + CCPP_PHYS_SUITE="FV3_GFS_v15p2" FCST_LEN_HRS="48" LBC_SPEC_INTVL_HRS="6" @@ -42,6 +45,8 @@ RUN_TASK_GET_OBS_MRMS="FALSE" RUN_TASK_GET_OBS_NDAS="FALSE" RUN_TASK_VX_GRIDSTAT="FALSE" RUN_TASK_VX_POINTSTAT="FALSE" +RUN_TASK_VX_ENSGRID="FALSE" +RUN_TASK_VX_ENSPOINT="FALSE" # # Uncomment the following line in order to use user-staged external model diff --git a/ush/config_defaults.sh b/ush/config_defaults.sh index 8fb02f5c9..72c758df4 100644 --- a/ush/config_defaults.sh +++ b/ush/config_defaults.sh @@ -648,7 +648,6 @@ NOMADS_file_type="nemsio" # directory or the cycle directories under it. # #----------------------------------------------------------------------- -# CCPP_PHYS_SUITE="FV3_GFS_v15p2" # #----------------------------------------------------------------------- @@ -1148,6 +1147,25 @@ VX_GRIDSTAT_03h_TN="run_gridstatvx_03h" VX_GRIDSTAT_06h_TN="run_gridstatvx_06h" VX_GRIDSTAT_24h_TN="run_gridstatvx_24h" VX_POINTSTAT_TN="run_pointstatvx" +VX_ENSGRID_TN="run_ensgridvx" +VX_ENSGRID_03h_TN="run_ensgridvx_03h" +VX_ENSGRID_06h_TN="run_ensgridvx_06h" +VX_ENSGRID_24h_TN="run_ensgridvx_24h" +VX_ENSGRID_REFC_TN="run_ensgridvx_refc" +VX_ENSGRID_RETOP_TN="run_ensgridvx_retop" +VX_ENSGRID_MEAN_TN="run_ensgridvx_mean" +VX_ENSGRID_PROB_TN="run_ensgridvx_prob" +VX_ENSGRID_MEAN_03h_TN="run_ensgridvx_mean_03h" +VX_ENSGRID_PROB_03h_TN="run_ensgridvx_prob_03h" +VX_ENSGRID_MEAN_06h_TN="run_ensgridvx_mean_06h" +VX_ENSGRID_PROB_06h_TN="run_ensgridvx_prob_06h" +VX_ENSGRID_MEAN_24h_TN="run_ensgridvx_mean_24h" +VX_ENSGRID_PROB_24h_TN="run_ensgridvx_prob_24h" +VX_ENSGRID_PROB_REFC_TN="run_ensgridvx_prob_refc" +VX_ENSGRID_PROB_RETOP_TN="run_ensgridvx_prob_retop" +VX_ENSPOINT_TN="run_enspointvx" +VX_ENSPOINT_MEAN_TN="run_enspointvx_mean" +VX_ENSPOINT_PROB_TN="run_enspointvx_prob" # #----------------------------------------------------------------------- # @@ -1191,6 +1209,15 @@ VX_POINTSTAT_TN="run_pointstatvx" # Flag that determines whether the point-stat verification task is to be # run. # +# RUN_TASK_VX_ENSGRID: +# Flag that determines whether the ensemble-stat verification for gridded +# data task is to be run. +# +# RUN_TASK_VX_ENSPOINT: +# Flag that determines whether the ensemble point verification task is +# to be run. If this flag is set, both ensemble-stat point verification +# and point verification of ensemble-stat output is computed. +# #----------------------------------------------------------------------- # RUN_TASK_MAKE_GRID="TRUE" @@ -1213,6 +1240,10 @@ RUN_TASK_GET_OBS_NDAS="FALSE" RUN_TASK_VX_GRIDSTAT="FALSE" RUN_TASK_VX_POINTSTAT="FALSE" + +RUN_TASK_VX_ENSGRID="FALSE" + +RUN_TASK_VX_ENSPOINT="FALSE" # #----------------------------------------------------------------------- # @@ -1421,6 +1452,12 @@ NNODES_GET_OBS_MRMS="1" NNODES_GET_OBS_NDAS="1" NNODES_VX_GRIDSTAT="1" NNODES_VX_POINTSTAT="1" +NNODES_VX_ENSGRID="1" +NNODES_VX_ENSGRID_MEAN="1" +NNODES_VX_ENSGRID_PROB="1" +NNODES_VX_ENSPOINT="1" +NNODES_VX_ENSPOINT_MEAN="1" +NNODES_VX_ENSPOINT_PROB="1" # # Number of MPI processes per node. # @@ -1438,6 +1475,12 @@ PPN_GET_OBS_MRMS="1" PPN_GET_OBS_NDAS="1" PPN_VX_GRIDSTAT="1" PPN_VX_POINTSTAT="1" +PPN_VX_ENSGRID="1" +PPN_VX_ENSGRID_MEAN="1" +PPN_VX_ENSGRID_PROB="1" +PPN_VX_ENSPOINT="1" +PPN_VX_ENSPOINT_MEAN="1" +PPN_VX_ENSPOINT_PROB="1" # # Walltimes. # @@ -1455,6 +1498,12 @@ WTIME_GET_OBS_MRMS="00:45:00" WTIME_GET_OBS_NDAS="02:00:00" WTIME_VX_GRIDSTAT="02:00:00" WTIME_VX_POINTSTAT="01:00:00" +WTIME_VX_ENSGRID="01:00:00" +WTIME_VX_ENSGRID_MEAN="01:00:00" +WTIME_VX_ENSGRID_PROB="01:00:00" +WTIME_VX_ENSPOINT="01:00:00" +WTIME_VX_ENSPOINT_MEAN="01:00:00" +WTIME_VX_ENSPOINT_PROB="01:00:00" # # Maximum number of attempts. # @@ -1477,6 +1526,25 @@ MAXTRIES_VX_GRIDSTAT_03h="1" MAXTRIES_VX_GRIDSTAT_06h="1" MAXTRIES_VX_GRIDSTAT_24h="1" MAXTRIES_VX_POINTSTAT="1" +MAXTRIES_VX_ENSGRID="1" +MAXTRIES_VX_ENSGRID_REFC="1" +MAXTRIES_VX_ENSGRID_RETOP="1" +MAXTRIES_VX_ENSGRID_03h="1" +MAXTRIES_VX_ENSGRID_06h="1" +MAXTRIES_VX_ENSGRID_24h="1" +MAXTRIES_VX_ENSGRID_MEAN="1" +MAXTRIES_VX_ENSGRID_PROB="1" +MAXTRIES_VX_ENSGRID_MEAN_03h="1" +MAXTRIES_VX_ENSGRID_PROB_03h="1" +MAXTRIES_VX_ENSGRID_MEAN_06h="1" +MAXTRIES_VX_ENSGRID_PROB_06h="1" +MAXTRIES_VX_ENSGRID_MEAN_24h="1" +MAXTRIES_VX_ENSGRID_PROB_24h="1" +MAXTRIES_VX_ENSGRID_PROB_REFC="1" +MAXTRIES_VX_ENSGRID_PROB_RETOP="1" +MAXTRIES_VX_ENSPOINT="1" +MAXTRIES_VX_ENSPOINT_MEAN="1" +MAXTRIES_VX_ENSPOINT_PROB="1" # #----------------------------------------------------------------------- # diff --git a/ush/generate_FV3LAM_wflow.sh b/ush/generate_FV3LAM_wflow.sh index 07aee4036..e5d2f496e 100755 --- a/ush/generate_FV3LAM_wflow.sh +++ b/ush/generate_FV3LAM_wflow.sh @@ -193,6 +193,25 @@ settings="\ 'vx_gridstat_06h_tn': ${VX_GRIDSTAT_06h_TN} 'vx_gridstat_24h_tn': ${VX_GRIDSTAT_24h_TN} 'vx_pointstat_tn': ${VX_POINTSTAT_TN} + 'vx_ensgrid_tn': ${VX_ENSGRID_TN} + 'vx_ensgrid_refc_tn': ${VX_ENSGRID_REFC_TN} + 'vx_ensgrid_retop_tn': ${VX_ENSGRID_RETOP_TN} + 'vx_ensgrid_03h_tn': ${VX_ENSGRID_03h_TN} + 'vx_ensgrid_06h_tn': ${VX_ENSGRID_06h_TN} + 'vx_ensgrid_24h_tn': ${VX_ENSGRID_24h_TN} + 'vx_ensgrid_mean_tn': ${VX_ENSGRID_MEAN_TN} + 'vx_ensgrid_prob_tn': ${VX_ENSGRID_PROB_TN} + 'vx_ensgrid_mean_03h_tn': ${VX_ENSGRID_MEAN_03h_TN} + 'vx_ensgrid_prob_03h_tn': ${VX_ENSGRID_PROB_03h_TN} + 'vx_ensgrid_mean_06h_tn': ${VX_ENSGRID_MEAN_06h_TN} + 'vx_ensgrid_prob_06h_tn': ${VX_ENSGRID_PROB_06h_TN} + 'vx_ensgrid_mean_24h_tn': ${VX_ENSGRID_MEAN_24h_TN} + 'vx_ensgrid_prob_24h_tn': ${VX_ENSGRID_PROB_24h_TN} + 'vx_ensgrid_prob_refc_tn': ${VX_ENSGRID_PROB_REFC_TN} + 'vx_ensgrid_prob_retop_tn': ${VX_ENSGRID_PROB_RETOP_TN} + 'vx_enspoint_tn': ${VX_ENSPOINT_TN} + 'vx_enspoint_mean_tn': ${VX_ENSPOINT_MEAN_TN} + 'vx_enspoint_prob_tn': ${VX_ENSPOINT_PROB_TN} # # Entity used to load the module file for each GET_OBS_* task. # @@ -214,6 +233,12 @@ settings="\ 'nnodes_get_obs_ndas': ${NNODES_GET_OBS_NDAS} 'nnodes_vx_gridstat': ${NNODES_VX_GRIDSTAT} 'nnodes_vx_pointstat': ${NNODES_VX_POINTSTAT} + 'nnodes_vx_ensgrid': ${NNODES_VX_ENSGRID} + 'nnodes_vx_ensgrid_mean': ${NNODES_VX_ENSGRID_MEAN} + 'nnodes_vx_ensgrid_prob': ${NNODES_VX_ENSGRID_PROB} + 'nnodes_vx_enspoint': ${NNODES_VX_ENSPOINT} + 'nnodes_vx_enspoint_mean': ${NNODES_VX_ENSPOINT_MEAN} + 'nnodes_vx_enspoint_prob': ${NNODES_VX_ENSPOINT_PROB} # # Number of cores used for a task # @@ -237,6 +262,12 @@ settings="\ 'ppn_get_obs_ndas': ${PPN_GET_OBS_NDAS} 'ppn_vx_gridstat': ${PPN_VX_GRIDSTAT} 'ppn_vx_pointstat': ${PPN_VX_POINTSTAT} + 'ppn_vx_ensgrid': ${PPN_VX_ENSGRID} + 'ppn_vx_ensgrid_mean': ${PPN_VX_ENSGRID_MEAN} + 'ppn_vx_ensgrid_prob': ${PPN_VX_ENSGRID_PROB} + 'ppn_vx_enspoint': ${PPN_VX_ENSPOINT} + 'ppn_vx_enspoint_mean': ${PPN_VX_ENSPOINT_MEAN} + 'ppn_vx_enspoint_prob': ${PPN_VX_ENSPOINT_PROB} # # Maximum wallclock time for each task. # @@ -254,6 +285,12 @@ settings="\ 'wtime_get_obs_ndas': ${WTIME_GET_OBS_NDAS} 'wtime_vx_gridstat': ${WTIME_VX_GRIDSTAT} 'wtime_vx_pointstat': ${WTIME_VX_POINTSTAT} + 'wtime_vx_ensgrid': ${WTIME_VX_ENSGRID} + 'wtime_vx_ensgrid_mean': ${WTIME_VX_ENSGRID_MEAN} + 'wtime_vx_ensgrid_prob': ${WTIME_VX_ENSGRID_PROB} + 'wtime_vx_enspoint': ${WTIME_VX_ENSPOINT} + 'wtime_vx_enspoint_mean': ${WTIME_VX_ENSPOINT_MEAN} + 'wtime_vx_enspoint_prob': ${WTIME_VX_ENSPOINT_PROB} # # Maximum number of tries for each task. # @@ -276,6 +313,25 @@ settings="\ 'maxtries_vx_gridstat_06h': ${MAXTRIES_VX_GRIDSTAT_06h} 'maxtries_vx_gridstat_24h': ${MAXTRIES_VX_GRIDSTAT_24h} 'maxtries_vx_pointstat': ${MAXTRIES_VX_POINTSTAT} + 'maxtries_vx_ensgrid': ${MAXTRIES_VX_ENSGRID} + 'maxtries_vx_ensgrid_refc': ${MAXTRIES_VX_ENSGRID_REFC} + 'maxtries_vx_ensgrid_retop': ${MAXTRIES_VX_ENSGRID_RETOP} + 'maxtries_vx_ensgrid_03h': ${MAXTRIES_VX_ENSGRID_03h} + 'maxtries_vx_ensgrid_06h': ${MAXTRIES_VX_ENSGRID_06h} + 'maxtries_vx_ensgrid_24h': ${MAXTRIES_VX_ENSGRID_24h} + 'maxtries_vx_ensgrid_mean': ${MAXTRIES_VX_ENSGRID_MEAN} + 'maxtries_vx_ensgrid_prob': ${MAXTRIES_VX_ENSGRID_PROB} + 'maxtries_vx_ensgrid_mean_03h': ${MAXTRIES_VX_ENSGRID_MEAN_03h} + 'maxtries_vx_ensgrid_prob_03h': ${MAXTRIES_VX_ENSGRID_PROB_03h} + 'maxtries_vx_ensgrid_mean_06h': ${MAXTRIES_VX_ENSGRID_MEAN_06h} + 'maxtries_vx_ensgrid_prob_06h': ${MAXTRIES_VX_ENSGRID_PROB_06h} + 'maxtries_vx_ensgrid_mean_24h': ${MAXTRIES_VX_ENSGRID_MEAN_24h} + 'maxtries_vx_ensgrid_prob_24h': ${MAXTRIES_VX_ENSGRID_PROB_24h} + 'maxtries_vx_ensgrid_prob_refc': ${MAXTRIES_VX_ENSGRID_PROB_REFC} + 'maxtries_vx_ensgrid_prob_retop': ${MAXTRIES_VX_ENSGRID_PROB_RETOP} + 'maxtries_vx_enspoint': ${MAXTRIES_VX_ENSPOINT} + 'maxtries_vx_enspoint_mean': ${MAXTRIES_VX_ENSPOINT_MEAN} + 'maxtries_vx_enspoint_prob': ${MAXTRIES_VX_ENSPOINT_PROB} # # Flags that specify whether to run the preprocessing or # verification-related tasks. @@ -289,6 +345,8 @@ settings="\ 'run_task_get_obs_ndas': ${RUN_TASK_GET_OBS_NDAS} 'run_task_vx_gridstat': ${RUN_TASK_VX_GRIDSTAT} 'run_task_vx_pointstat': ${RUN_TASK_VX_POINTSTAT} + 'run_task_vx_ensgrid': ${RUN_TASK_VX_ENSGRID} + 'run_task_vx_enspoint': ${RUN_TASK_VX_ENSPOINT} # # Number of physical cores per node for the current machine. # diff --git a/ush/setup.sh b/ush/setup.sh index ce2dacc61..bc11c67b4 100755 --- a/ush/setup.sh +++ b/ush/setup.sh @@ -290,6 +290,49 @@ elif [ "${RUN_TASK_VX_POINTSTAT}" = "FALSE" ] || \ [ "${RUN_TASK_VX_POINTSTAT}" = "NO" ]; then RUN_TASK_VX_POINTSTAT="FALSE" fi + +# +#----------------------------------------------------------------------- +# +# Make sure that RUN_TASK_VX_ENSGRID is set to a valid value. +# +#----------------------------------------------------------------------- +# +check_var_valid_value "RUN_TASK_VX_ENSGRID" "valid_vals_RUN_TASK_VX_ENSGRID" +# +# Set RUN_TASK_VX_ENSGRID to either "TRUE" or "FALSE" so we don't have to +# consider other valid values later on. +# +RUN_TASK_VX_ENSGRID=${RUN_TASK_VX_ENSGRID^^} +if [ "${RUN_TASK_VX_ENSGRID}" = "TRUE" ] || \ + [ "${RUN_TASK_VX_ENSGRID}" = "YES" ]; then + RUN_TASK_VX_ENSGRID="TRUE" +elif [ "${RUN_TASK_VX_ENSGRID}" = "FALSE" ] || \ + [ "${RUN_TASK_VX_ENSGRID}" = "NO" ]; then + RUN_TASK_VX_ENSGRID="FALSE" +fi + +# +# +#----------------------------------------------------------------------- +# +# Make sure that RUN_TASK_VX_ENSPOINT is set to a valid value. +# +#----------------------------------------------------------------------- +# +check_var_valid_value "RUN_TASK_VX_ENSPOINT" "valid_vals_RUN_TASK_VX_ENSPOINT" +# +# Set RUN_TASK_VX_ENSPOINT to either "TRUE" or "FALSE" so we don't have to +# consider other valid values later on. +# +RUN_TASK_VX_ENSPOINT=${RUN_TASK_VX_ENSPOINT^^} +if [ "${RUN_TASK_VX_ENSPOINT}" = "TRUE" ] || \ + [ "${RUN_TASK_VX_ENSPOINT}" = "YES" ]; then + RUN_TASK_VX_ENSPOINT="TRUE" +elif [ "${RUN_TASK_VX_ENSPOINT}" = "FALSE" ] || \ + [ "${RUN_TASK_VX_ENSPOINT}" = "NO" ]; then + RUN_TASK_VX_ENSPOINT="FALSE" +fi # #----------------------------------------------------------------------- # @@ -1723,7 +1766,6 @@ Reset values are: msg=" When RUN_ENVIR is set to \"nco\", the workflow assumes that pregenerated orography files already exist in the directory - \${FIXLAM_NCO_BASEDIR}/\${PREDEF_GRID_NAME} where @@ -1834,6 +1876,26 @@ Reset value is:" fi + if [ "${RUN_TASK_VX_ENSGRID}" = "TRUE" ] || \ + [ "${RUN_TASK_VX_ENSGRID}" = "FALSE" ]; then + + msg=" +When RUN_ENVIR is set to \"nco\", it is assumed that the verification +will not be run. + RUN_TASK_VX_ENSGRID = \"${RUN_TASK_VX_ENSGRID}\" +Resetting RUN_TASK_VX_ENSGRID to \"FALSE\" +Reset value is:" + + RUN_TASK_VX_ENSGRID="FALSE" + + msg="$msg"" + RUN_TASK_VX_ENSGRID = \"${RUN_TASK_VX_ENSGRID}\" +" + + print_info_msg "$msg" + + fi + # #----------------------------------------------------------------------- # diff --git a/ush/templates/FV3LAM_wflow.xml b/ush/templates/FV3LAM_wflow.xml index 519855266..f7e7b9471 100644 --- a/ush/templates/FV3LAM_wflow.xml +++ b/ush/templates/FV3LAM_wflow.xml @@ -22,26 +22,45 @@ Parameters needed by the job scheduler. - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + - - - - + + + + + + + + + + + + + + + + + + + + + + + @@ -666,7 +685,6 @@ the tag to be identical to the ones above for other output times. ************************************************************************ ************************************************************************ --> - &RSRV_HPSS; @@ -692,14 +710,6 @@ the tag to be identical to the ones above for other output times. FHR {% for h in range(0, fcst_len_hrs+1) %}{{ " %02d" % h }}{% endfor %} ACCUM01 - - {%- if write_dopost %} - - {%- else %} - - {%- endif %} - - {%- endif %} @@ -708,7 +718,6 @@ the tag to be identical to the ones above for other output times. ************************************************************************ ************************************************************************ --> - &RSRV_HPSS; @@ -735,14 +744,6 @@ the tag to be identical to the ones above for other output times. SCRIPTSDIR&SCRIPTSDIR; VARREFC RETOP - - {%- if write_dopost %} - - {%- else %} - - {%- endif %} - - {%- endif %} @@ -751,7 +752,6 @@ the tag to be identical to the ones above for other output times. ************************************************************************ ************************************************************************ --> - &RSRV_HPSS; @@ -776,14 +776,6 @@ the tag to be identical to the ones above for other output times. cyc@H FHR {% for h in range(0, fcst_len_hrs+1) %}{{ " %02d" % h }}{% endfor %} - - {%- if write_dopost %} - - {%- else %} - - {%- endif %} - - {%- endif %} @@ -792,8 +784,7 @@ the tag to be identical to the ones above for other output times. ************************************************************************ ************************************************************************ --> - - + &RSRV_DEFAULT; &LOAD_MODULES_RUN_TASK_FP; "&VX_TN;" "&JOBSDIR;/JREGIONAL_RUN_VX_GRIDSTAT" @@ -801,7 +792,7 @@ the tag to be identical to the ones above for other output times. {{ wtime_vx_gridstat }} &NCORES_PER_NODE; &VX_GRIDSTAT_TN; - &LOGDIR;/&VX_GRIDSTAT_TN;_@Y@m@d@H.log + &LOGDIR;/&VX_GRIDSTAT_TN;{{ uscore_ensmem_name }}_@Y@m@d@H.log GLOBAL_VAR_DEFNS_FP&GLOBAL_VAR_DEFNS_FP; OBS_DIR&CCPA_OBS_DIR; @@ -812,10 +803,20 @@ the tag to be identical to the ones above for other output times. FHR {% for h in range(1, fcst_len_hrs+1) %}{{ " %02d" % h }}{% endfor %} VARAPCP ACCUM01 + {%- if do_ensemble %} + SLASH_ENSMEM_SUBDIR{{ slash_ensmem_subdir }} + {%- endif %} {%- if run_task_get_obs_ccpa %} - + + + {%- if write_dopost %} + + {%- else %} + + {%- endif %} + {%- else %} {%- if write_dopost %} @@ -833,7 +834,7 @@ the tag to be identical to the ones above for other output times. ************************************************************************ ************************************************************************ --> - + &RSRV_DEFAULT; &LOAD_MODULES_RUN_TASK_FP; "&VX_TN;" "&JOBSDIR;/JREGIONAL_RUN_VX_GRIDSTAT" @@ -841,7 +842,7 @@ the tag to be identical to the ones above for other output times. {{ wtime_vx_gridstat }} &NCORES_PER_NODE; &VX_GRIDSTAT_REFC_TN; - &LOGDIR;/&VX_GRIDSTAT_REFC_TN;_@Y@m@d@H.log + &LOGDIR;/&VX_GRIDSTAT_REFC_TN;{{ uscore_ensmem_name }}_@Y@m@d@H.log GLOBAL_VAR_DEFNS_FP&GLOBAL_VAR_DEFNS_FP; OBS_DIR&MRMS_OBS_DIR; @@ -851,10 +852,20 @@ the tag to be identical to the ones above for other output times. cyc@H FHR {% for h in range(1, fcst_len_hrs+1) %}{{ " %02d" % h }}{% endfor %} VARREFC + {%- if do_ensemble %} + SLASH_ENSMEM_SUBDIR{{ slash_ensmem_subdir }} + {%- endif %} {%- if run_task_get_obs_mrms %} - + + + {%- if write_dopost %} + + {%- else %} + + {%- endif %} + {%- else %} {%- if write_dopost %} @@ -872,7 +883,7 @@ the tag to be identical to the ones above for other output times. ************************************************************************ ************************************************************************ --> - + &RSRV_DEFAULT; &LOAD_MODULES_RUN_TASK_FP; "&VX_TN;" "&JOBSDIR;/JREGIONAL_RUN_VX_GRIDSTAT" @@ -880,7 +891,7 @@ the tag to be identical to the ones above for other output times. {{ wtime_vx_gridstat }} &NCORES_PER_NODE; &VX_GRIDSTAT_RETOP_TN; - &LOGDIR;/&VX_GRIDSTAT_RETOP_TN;_@Y@m@d@H.log + &LOGDIR;/&VX_GRIDSTAT_RETOP_TN;{{ uscore_ensmem_name }}_@Y@m@d@H.log GLOBAL_VAR_DEFNS_FP&GLOBAL_VAR_DEFNS_FP; OBS_DIR&MRMS_OBS_DIR; @@ -890,10 +901,20 @@ the tag to be identical to the ones above for other output times. cyc@H FHR {% for h in range(1, fcst_len_hrs+1) %}{{ " %02d" % h }}{% endfor %} VARRETOP + {%- if do_ensemble %} + SLASH_ENSMEM_SUBDIR{{ slash_ensmem_subdir }} + {%- endif %} {%- if run_task_get_obs_mrms %} - + + + {%- if write_dopost %} + + {%- else %} + + {%- endif %} + {%- else %} {%- if write_dopost %} @@ -906,12 +927,12 @@ the tag to be identical to the ones above for other output times. {%- endif %} -{%- if run_task_vx_gridstat %} +{%- if run_task_vx_gridstat and fcst_len_hrs >= 3 %} - + &RSRV_DEFAULT; &LOAD_MODULES_RUN_TASK_FP; "&VX_TN;" "&JOBSDIR;/JREGIONAL_RUN_VX_GRIDSTAT" @@ -919,7 +940,7 @@ the tag to be identical to the ones above for other output times. {{ wtime_vx_gridstat }} &NCORES_PER_NODE; &VX_GRIDSTAT_03h_TN; - &LOGDIR;/&VX_GRIDSTAT_03h_TN;_@Y@m@d@H.log + &LOGDIR;/&VX_GRIDSTAT_03h_TN;{{ uscore_ensmem_name }}_@Y@m@d@H.log GLOBAL_VAR_DEFNS_FP&GLOBAL_VAR_DEFNS_FP; OBS_DIR&CCPA_OBS_DIR; @@ -930,20 +951,23 @@ the tag to be identical to the ones above for other output times. FHR {% for h in range(3, fcst_len_hrs+1, 3) %}{{ " %02d" % h }}{% endfor %} VARAPCP ACCUM03 + {%- if do_ensemble %} + SLASH_ENSMEM_SUBDIR{{ slash_ensmem_subdir }} + {%- endif %} - + {%- endif %} -{%- if run_task_vx_gridstat %} +{%- if run_task_vx_gridstat and fcst_len_hrs >= 6 %} - + &RSRV_DEFAULT; &LOAD_MODULES_RUN_TASK_FP; "&VX_TN;" "&JOBSDIR;/JREGIONAL_RUN_VX_GRIDSTAT" @@ -951,7 +975,7 @@ the tag to be identical to the ones above for other output times. {{ wtime_vx_gridstat }} &NCORES_PER_NODE; &VX_GRIDSTAT_06h_TN; - &LOGDIR;/&VX_GRIDSTAT_06h_TN;_@Y@m@d@H.log + &LOGDIR;/&VX_GRIDSTAT_06h_TN;{{ uscore_ensmem_name }}_@Y@m@d@H.log GLOBAL_VAR_DEFNS_FP&GLOBAL_VAR_DEFNS_FP; OBS_DIR&CCPA_OBS_DIR; @@ -962,20 +986,23 @@ the tag to be identical to the ones above for other output times. FHR {% for h in range(6, fcst_len_hrs+1, 6) %}{{ " %02d" % h }}{% endfor %} VARAPCP ACCUM06 + {%- if do_ensemble %} + SLASH_ENSMEM_SUBDIR{{ slash_ensmem_subdir }} + {%- endif %} - + {%- endif %} -{%- if run_task_vx_gridstat %} +{%- if run_task_vx_gridstat and fcst_len_hrs >= 24 %} - + &RSRV_DEFAULT; &LOAD_MODULES_RUN_TASK_FP; "&VX_TN;" "&JOBSDIR;/JREGIONAL_RUN_VX_GRIDSTAT" @@ -983,7 +1010,7 @@ the tag to be identical to the ones above for other output times. {{ wtime_vx_gridstat }} &NCORES_PER_NODE; &VX_GRIDSTAT_24h_TN; - &LOGDIR;/&VX_GRIDSTAT_24h_TN;_@Y@m@d@H.log + &LOGDIR;/&VX_GRIDSTAT_24h_TN;{{ uscore_ensmem_name }}_@Y@m@d@H.log GLOBAL_VAR_DEFNS_FP&GLOBAL_VAR_DEFNS_FP; OBS_DIR&CCPA_OBS_DIR; @@ -994,9 +1021,12 @@ the tag to be identical to the ones above for other output times. FHR {% for h in range(24, fcst_len_hrs+1, 24) %}{{ " %02d" % h }}{% endfor %} VARAPCP ACCUM24 + {%- if do_ensemble %} + SLASH_ENSMEM_SUBDIR{{ slash_ensmem_subdir }} + {%- endif %} - + @@ -1007,15 +1037,15 @@ the tag to be identical to the ones above for other output times. ************************************************************************ ************************************************************************ --> - - + &RSRV_DEFAULT; + &LOAD_MODULES_RUN_TASK_FP; "&VX_TN;" "&JOBSDIR;/JREGIONAL_RUN_VX_POINTSTAT" {{ nnodes_vx_pointstat }}:ppn={{ ppn_vx_pointstat }} {{ wtime_vx_pointstat }} &NCORES_PER_NODE; &VX_POINTSTAT_TN; - &LOGDIR;/&VX_POINTSTAT_TN;_@Y@m@d@H.log + &LOGDIR;/&VX_POINTSTAT_TN;{{ uscore_ensmem_name }}_@Y@m@d@H.log GLOBAL_VAR_DEFNS_FP&GLOBAL_VAR_DEFNS_FP; OBS_DIR&NDAS_OBS_DIR; @@ -1023,11 +1053,21 @@ the tag to be identical to the ones above for other output times. CDATE@Y@m@d@H PDY@Y@m@d cyc@H - FHR {%- for h in range(0, fcst_len_hrs+1) %}{{ " %02d" % h }}{% endfor %} + FHR {% for h in range(0, fcst_len_hrs+1) %}{{ " %02d" % h }}{% endfor %} + {%- if do_ensemble %} + SLASH_ENSMEM_SUBDIR{{ slash_ensmem_subdir }} + {%- endif %} {%- if run_task_get_obs_ndas %} - + + + {%- if write_dopost %} + + {%- else %} + + {%- endif %} + {%- else %} {%- if write_dopost %} @@ -1038,9 +1078,593 @@ the tag to be identical to the ones above for other output times. - {%- endif %} + {%- if do_ensemble %} {%- endif %} + +{%- if run_task_vx_ensgrid %} + + + + &RSRV_DEFAULT; + &LOAD_MODULES_RUN_TASK_FP; "&VX_TN;" "&JOBSDIR;/JREGIONAL_RUN_VX_ENSGRID" + {{ nnodes_vx_ensgrid }}:ppn={{ ppn_vx_ensgrid }} + {{ wtime_vx_ensgrid }} + &NCORES_PER_NODE; + &VX_ENSGRID_TN; + &LOGDIR;/&VX_ENSGRID_TN;_@Y@m@d@H.log + + GLOBAL_VAR_DEFNS_FP&GLOBAL_VAR_DEFNS_FP; + OBS_DIR&CCPA_OBS_DIR; + CYCLE_DIR&CYCLE_BASEDIR;/@Y@m@d@H + CDATE@Y@m@d@H + PDY@Y@m@d + cyc@H + FHR {% for h in range(1, fcst_len_hrs+1) %}{{ " %02d" % h }}{% endfor %} + VARAPCP + ACCUM01 + + + + + + + +{%- if fcst_len_hrs >= 3 %} + + + + + &RSRV_DEFAULT; + &LOAD_MODULES_RUN_TASK_FP; "&VX_TN;" "&JOBSDIR;/JREGIONAL_RUN_VX_ENSGRID" + {{ nnodes_vx_ensgrid }}:ppn={{ ppn_vx_ensgrid }} + {{ wtime_vx_ensgrid }} + &NCORES_PER_NODE; + &VX_ENSGRID_03h_TN; + &LOGDIR;/&VX_ENSGRID_03h_TN;_@Y@m@d@H.log + + GLOBAL_VAR_DEFNS_FP&GLOBAL_VAR_DEFNS_FP; + OBS_DIR&CCPA_OBS_DIR; + CYCLE_DIR&CYCLE_BASEDIR;/@Y@m@d@H + CDATE@Y@m@d@H + PDY@Y@m@d + cyc@H + FHR {% for h in range(3, fcst_len_hrs+1, 3) %}{{ " %02d" % h }}{% endfor %} + VARAPCP + ACCUM03 + + + + + + +{%- endif %} +{%- if fcst_len_hrs >= 6 %} + + + + + &RSRV_DEFAULT; + &LOAD_MODULES_RUN_TASK_FP; "&VX_TN;" "&JOBSDIR;/JREGIONAL_RUN_VX_ENSGRID" + {{ nnodes_vx_ensgrid }}:ppn={{ ppn_vx_ensgrid }} + {{ wtime_vx_ensgrid }} + &NCORES_PER_NODE; + &VX_ENSGRID_06h_TN; + &LOGDIR;/&VX_ENSGRID_06h_TN;_@Y@m@d@H.log + + GLOBAL_VAR_DEFNS_FP&GLOBAL_VAR_DEFNS_FP; + OBS_DIR&CCPA_OBS_DIR; + CYCLE_DIR&CYCLE_BASEDIR;/@Y@m@d@H + CDATE@Y@m@d@H + PDY@Y@m@d + cyc@H + FHR {% for h in range(6, fcst_len_hrs+1, 6) %}{{ " %02d" % h }}{% endfor %} + VARAPCP + ACCUM06 + + + + + + +{%- endif %} +{%- if fcst_len_hrs >= 24 %} + + + + + &RSRV_DEFAULT; + &LOAD_MODULES_RUN_TASK_FP; "&VX_TN;" "&JOBSDIR;/JREGIONAL_RUN_VX_ENSGRID" + {{ nnodes_vx_ensgrid }}:ppn={{ ppn_vx_ensgrid }} + {{ wtime_vx_ensgrid }} + &NCORES_PER_NODE; + &VX_ENSGRID_24h_TN; + &LOGDIR;/&VX_ENSGRID_24h_TN;_@Y@m@d@H.log + + GLOBAL_VAR_DEFNS_FP&GLOBAL_VAR_DEFNS_FP; + OBS_DIR&CCPA_OBS_DIR; + CYCLE_DIR&CYCLE_BASEDIR;/@Y@m@d@H + CDATE@Y@m@d@H + PDY@Y@m@d + cyc@H + FHR {% for h in range(24, fcst_len_hrs+1, 24) %}{{ " %02d" % h }}{% endfor %} + VARAPCP + ACCUM24 + + + + + + +{%- endif %} + + + + + &RSRV_DEFAULT; + &LOAD_MODULES_RUN_TASK_FP; "&VX_TN;" "&JOBSDIR;/JREGIONAL_RUN_VX_ENSGRID" + {{ nnodes_vx_ensgrid }}:ppn={{ ppn_vx_ensgrid }} + {{ wtime_vx_ensgrid }} + &NCORES_PER_NODE; + &VX_ENSGRID_REFC_TN; + &LOGDIR;/&VX_ENSGRID_REFC_TN;_@Y@m@d@H.log + + GLOBAL_VAR_DEFNS_FP&GLOBAL_VAR_DEFNS_FP; + OBS_DIR&MRMS_OBS_DIR; + CYCLE_DIR&CYCLE_BASEDIR;/@Y@m@d@H + CDATE@Y@m@d@H + PDY@Y@m@d + cyc@H + FHR {% for h in range(1, fcst_len_hrs+1) %}{{ " %02d" % h }}{% endfor %} + VARREFC + + + + + + + + + + + + &RSRV_DEFAULT; + &LOAD_MODULES_RUN_TASK_FP; "&VX_TN;" "&JOBSDIR;/JREGIONAL_RUN_VX_ENSGRID" + {{ nnodes_vx_ensgrid }}:ppn={{ ppn_vx_ensgrid }} + {{ wtime_vx_ensgrid }} + &NCORES_PER_NODE; + &VX_ENSGRID_RETOP_TN; + &LOGDIR;/&VX_ENSGRID_RETOP_TN;_@Y@m@d@H.log + + GLOBAL_VAR_DEFNS_FP&GLOBAL_VAR_DEFNS_FP; + OBS_DIR&MRMS_OBS_DIR; + CYCLE_DIR&CYCLE_BASEDIR;/@Y@m@d@H + CDATE@Y@m@d@H + PDY@Y@m@d + cyc@H + FHR {% for h in range(1, fcst_len_hrs+1) %}{{ " %02d" % h }}{% endfor %} + VARRETOP + + + + + + + + + + + &RSRV_DEFAULT; + &LOAD_MODULES_RUN_TASK_FP; "&VX_TN;" "&JOBSDIR;/JREGIONAL_RUN_VX_ENSGRID_MEAN" + {{ nnodes_vx_ensgrid_mean }}:ppn={{ ppn_vx_ensgrid_mean }} + {{ wtime_vx_ensgrid_mean }} + &NCORES_PER_NODE; + &VX_ENSGRID_MEAN_TN; + &LOGDIR;/&VX_ENSGRID_MEAN_TN;_@Y@m@d@H.log + + GLOBAL_VAR_DEFNS_FP&GLOBAL_VAR_DEFNS_FP; + OBS_DIR&CCPA_OBS_DIR; + CYCLE_DIR&CYCLE_BASEDIR;/@Y@m@d@H + CDATE@Y@m@d@H + PDY@Y@m@d + cyc@H + FHR {% for h in range(1, fcst_len_hrs+1) %}{{ " %02d" % h }}{% endfor %} + VARAPCP + ACCUM01 + + + + + + + + + + + + &RSRV_DEFAULT; + &LOAD_MODULES_RUN_TASK_FP; "&VX_TN;" "&JOBSDIR;/JREGIONAL_RUN_VX_ENSGRID_PROB" + {{ nnodes_vx_ensgrid_prob }}:ppn={{ ppn_vx_ensgrid_prob }} + {{ wtime_vx_ensgrid_prob }} + &NCORES_PER_NODE; + &VX_ENSGRID_PROB_TN; + &LOGDIR;/&VX_ENSGRID_PROB_TN;_@Y@m@d@H.log + + GLOBAL_VAR_DEFNS_FP&GLOBAL_VAR_DEFNS_FP; + OBS_DIR&CCPA_OBS_DIR; + CYCLE_DIR&CYCLE_BASEDIR;/@Y@m@d@H + CDATE@Y@m@d@H + PDY@Y@m@d + cyc@H + FHR {% for h in range(1, fcst_len_hrs+1) %}{{ " %02d" % h }}{% endfor %} + VARAPCP + ACCUM01 + + + + + + +{%- if fcst_len_hrs >= 3 %} + + + + + &RSRV_DEFAULT; + &LOAD_MODULES_RUN_TASK_FP; "&VX_TN;" "&JOBSDIR;/JREGIONAL_RUN_VX_ENSGRID_MEAN" + {{ nnodes_vx_ensgrid_mean }}:ppn={{ ppn_vx_ensgrid_mean }} + {{ wtime_vx_ensgrid_mean }} + &NCORES_PER_NODE; + &VX_ENSGRID_MEAN_03h_TN; + &LOGDIR;/&VX_ENSGRID_MEAN_03h_TN;_@Y@m@d@H.log + + GLOBAL_VAR_DEFNS_FP&GLOBAL_VAR_DEFNS_FP; + OBS_DIR&CCPA_OBS_DIR; + CYCLE_DIR&CYCLE_BASEDIR;/@Y@m@d@H + CDATE@Y@m@d@H + PDY@Y@m@d + cyc@H + FHR {% for h in range(3, fcst_len_hrs+1, 3) %}{{ " %02d" % h }}{% endfor %} + VARAPCP + ACCUM03 + + + + + + + + + + + + &RSRV_DEFAULT; + &LOAD_MODULES_RUN_TASK_FP; "&VX_TN;" "&JOBSDIR;/JREGIONAL_RUN_VX_ENSGRID_PROB" + {{ nnodes_vx_ensgrid_prob }}:ppn={{ ppn_vx_ensgrid_prob }} + {{ wtime_vx_ensgrid_prob }} + &NCORES_PER_NODE; + &VX_ENSGRID_PROB_03h_TN; + &LOGDIR;/&VX_ENSGRID_PROB_03h_TN;_@Y@m@d@H.log + + GLOBAL_VAR_DEFNS_FP&GLOBAL_VAR_DEFNS_FP; + OBS_DIR&CCPA_OBS_DIR; + CYCLE_DIR&CYCLE_BASEDIR;/@Y@m@d@H + CDATE@Y@m@d@H + PDY@Y@m@d + cyc@H + FHR {% for h in range(3, fcst_len_hrs+1, 3) %}{{ " %02d" % h }}{% endfor %} + VARAPCP + ACCUM03 + + + + + + +{%- endif %} +{%- if fcst_len_hrs >= 6 %} + + + + + &RSRV_DEFAULT; + &LOAD_MODULES_RUN_TASK_FP; "&VX_TN;" "&JOBSDIR;/JREGIONAL_RUN_VX_ENSGRID_MEAN" + {{ nnodes_vx_ensgrid_mean }}:ppn={{ ppn_vx_ensgrid_mean }} + {{ wtime_vx_ensgrid_mean }} + &NCORES_PER_NODE; + &VX_ENSGRID_MEAN_06h_TN; + &LOGDIR;/&VX_ENSGRID_MEAN_06h_TN;_@Y@m@d@H.log + + GLOBAL_VAR_DEFNS_FP&GLOBAL_VAR_DEFNS_FP; + OBS_DIR&CCPA_OBS_DIR; + CYCLE_DIR&CYCLE_BASEDIR;/@Y@m@d@H + CDATE@Y@m@d@H + PDY@Y@m@d + cyc@H + FHR {% for h in range(6, fcst_len_hrs+1, 6) %}{{ " %02d" % h }}{% endfor %} + VARAPCP + ACCUM06 + + + + + + + + + + + + &RSRV_DEFAULT; + &LOAD_MODULES_RUN_TASK_FP; "&VX_TN;" "&JOBSDIR;/JREGIONAL_RUN_VX_ENSGRID_PROB" + {{ nnodes_vx_ensgrid_prob }}:ppn={{ ppn_vx_ensgrid_prob }} + {{ wtime_vx_ensgrid_prob }} + &NCORES_PER_NODE; + &VX_ENSGRID_PROB_06h_TN; + &LOGDIR;/&VX_ENSGRID_PROB_06h_TN;_@Y@m@d@H.log + + GLOBAL_VAR_DEFNS_FP&GLOBAL_VAR_DEFNS_FP; + OBS_DIR&CCPA_OBS_DIR; + CYCLE_DIR&CYCLE_BASEDIR;/@Y@m@d@H + CDATE@Y@m@d@H + PDY@Y@m@d + cyc@H + FHR {% for h in range(6, fcst_len_hrs+1, 6) %}{{ " %02d" % h }}{% endfor %} + VARAPCP + ACCUM06 + + + + + + +{%- endif %} +{%- if fcst_len_hrs >= 24 %} + + + + + &RSRV_DEFAULT; + &LOAD_MODULES_RUN_TASK_FP; "&VX_TN;" "&JOBSDIR;/JREGIONAL_RUN_VX_ENSGRID_MEAN" + {{ nnodes_vx_ensgrid_mean }}:ppn={{ ppn_vx_ensgrid_mean }} + {{ wtime_vx_ensgrid_mean }} + &NCORES_PER_NODE; + &VX_ENSGRID_MEAN_24h_TN; + &LOGDIR;/&VX_ENSGRID_MEAN_24h_TN;_@Y@m@d@H.log + + GLOBAL_VAR_DEFNS_FP&GLOBAL_VAR_DEFNS_FP; + OBS_DIR&CCPA_OBS_DIR; + CYCLE_DIR&CYCLE_BASEDIR;/@Y@m@d@H + CDATE@Y@m@d@H + PDY@Y@m@d + cyc@H + FHR {% for h in range(24, fcst_len_hrs+1, 24) %}{{ " %02d" % h }}{% endfor %} + VARAPCP + ACCUM24 + + + + + + + + + + + + &RSRV_DEFAULT; + &LOAD_MODULES_RUN_TASK_FP; "&VX_TN;" "&JOBSDIR;/JREGIONAL_RUN_VX_ENSGRID_PROB" + {{ nnodes_vx_ensgrid_prob }}:ppn={{ ppn_vx_ensgrid_prob }} + {{ wtime_vx_ensgrid_prob }} + &NCORES_PER_NODE; + &VX_ENSGRID_PROB_24h_TN; + &LOGDIR;/&VX_ENSGRID_PROB_24h_TN;_@Y@m@d@H.log + + GLOBAL_VAR_DEFNS_FP&GLOBAL_VAR_DEFNS_FP; + OBS_DIR&CCPA_OBS_DIR; + CYCLE_DIR&CYCLE_BASEDIR;/@Y@m@d@H + CDATE@Y@m@d@H + PDY@Y@m@d + cyc@H + FHR {% for h in range(24, fcst_len_hrs+1, 24) %}{{ " %02d" % h }}{% endfor %} + VARAPCP + ACCUM24 + + + + + + +{%- endif %} + + + + + &RSRV_DEFAULT; + &LOAD_MODULES_RUN_TASK_FP; "&VX_TN;" "&JOBSDIR;/JREGIONAL_RUN_VX_ENSGRID_PROB" + {{ nnodes_vx_ensgrid_prob }}:ppn={{ ppn_vx_ensgrid_prob }} + {{ wtime_vx_ensgrid_prob }} + &NCORES_PER_NODE; + &VX_ENSGRID_PROB_REFC_TN; + &LOGDIR;/&VX_ENSGRID_PROB_REFC_TN;_@Y@m@d@H.log + + GLOBAL_VAR_DEFNS_FP&GLOBAL_VAR_DEFNS_FP; + OBS_DIR&MRMS_OBS_DIR; + CYCLE_DIR&CYCLE_BASEDIR;/@Y@m@d@H + CDATE@Y@m@d@H + PDY@Y@m@d + cyc@H + FHR {% for h in range(1, fcst_len_hrs+1) %}{{ " %02d" % h }}{% endfor %} + VARREFC + + + + + + + + + + + + &RSRV_DEFAULT; + &LOAD_MODULES_RUN_TASK_FP; "&VX_TN;" "&JOBSDIR;/JREGIONAL_RUN_VX_ENSGRID_PROB" + {{ nnodes_vx_ensgrid_prob }}:ppn={{ ppn_vx_ensgrid_prob }} + {{ wtime_vx_ensgrid_prob }} + &NCORES_PER_NODE; + &VX_ENSGRID_PROB_RETOP_TN; + &LOGDIR;/&VX_ENSGRID_PROB_RETOP_TN;_@Y@m@d@H.log + + GLOBAL_VAR_DEFNS_FP&GLOBAL_VAR_DEFNS_FP; + OBS_DIR&MRMS_OBS_DIR; + CYCLE_DIR&CYCLE_BASEDIR;/@Y@m@d@H + CDATE@Y@m@d@H + PDY@Y@m@d + cyc@H + FHR {% for h in range(1, fcst_len_hrs+1) %}{{ " %02d" % h }}{% endfor %} + VARRETOP + + + + + + +{%- endif %} + +{%- if run_task_vx_enspoint %} + + + + &RSRV_DEFAULT; + &LOAD_MODULES_RUN_TASK_FP; "&VX_TN;" "&JOBSDIR;/JREGIONAL_RUN_VX_ENSPOINT" + {{ nnodes_vx_enspoint }}:ppn={{ ppn_vx_enspoint }} + {{ wtime_vx_enspoint }} + &NCORES_PER_NODE; + &VX_ENSPOINT_TN; + &LOGDIR;/&VX_ENSPOINT_TN;_@Y@m@d@H.log + + GLOBAL_VAR_DEFNS_FP&GLOBAL_VAR_DEFNS_FP; + OBS_DIR&NDAS_OBS_DIR; + CYCLE_DIR&CYCLE_BASEDIR;/@Y@m@d@H + CDATE@Y@m@d@H + PDY@Y@m@d + cyc@H + FHR {% for h in range(0, fcst_len_hrs+1) %}{{ " %02d" % h }}{% endfor %} + + + + + + + + + + + &RSRV_DEFAULT; + &LOAD_MODULES_RUN_TASK_FP; "&VX_TN;" "&JOBSDIR;/JREGIONAL_RUN_VX_ENSPOINT_MEAN" + {{ nnodes_vx_enspoint_mean }}:ppn={{ ppn_vx_enspoint_mean }} + {{ wtime_vx_enspoint_mean }} + &NCORES_PER_NODE; + &VX_ENSPOINT_MEAN_TN; + &LOGDIR;/&VX_ENSPOINT_MEAN_TN;_@Y@m@d@H.log + + GLOBAL_VAR_DEFNS_FP&GLOBAL_VAR_DEFNS_FP; + OBS_DIR&NDAS_OBS_DIR; + CYCLE_DIR&CYCLE_BASEDIR;/@Y@m@d@H + CDATE@Y@m@d@H + PDY@Y@m@d + cyc@H + FHR {% for h in range(0, fcst_len_hrs+1) %}{{ " %02d" % h }}{% endfor %} + + + + + + + + + + + &RSRV_DEFAULT; + &LOAD_MODULES_RUN_TASK_FP; "&VX_TN;" "&JOBSDIR;/JREGIONAL_RUN_VX_ENSPOINT_PROB" + {{ nnodes_vx_enspoint_prob }}:ppn={{ ppn_vx_enspoint_prob }} + {{ wtime_vx_enspoint_prob }} + &NCORES_PER_NODE; + &VX_ENSPOINT_PROB_TN; + &LOGDIR;/&VX_ENSPOINT_PROB_TN;_@Y@m@d@H.log + + GLOBAL_VAR_DEFNS_FP&GLOBAL_VAR_DEFNS_FP; + OBS_DIR&NDAS_OBS_DIR; + CYCLE_DIR&CYCLE_BASEDIR;/@Y@m@d@H + CDATE@Y@m@d@H + PDY@Y@m@d + cyc@H + FHR {% for h in range(0, fcst_len_hrs+1) %}{{ " %02d" % h }}{% endfor %} + + + + + + + +{%- endif %} diff --git a/ush/templates/parm/met/EnsembleStatConfig_APCP b/ush/templates/parm/met/EnsembleStatConfig_APCP new file mode 100755 index 000000000..34d560cb6 --- /dev/null +++ b/ush/templates/parm/met/EnsembleStatConfig_APCP @@ -0,0 +1,257 @@ +//////////////////////////////////////////////////////////////////////////////// +// +// Ensemble-Stat configuration file. +// +// For additional information, see the MET_BASE/config/README file. +// +//////////////////////////////////////////////////////////////////////////////// + +// +// Output model name to be written +// +model = "${MODEL}"; + +// +// Output description to be written +// May be set separately in each "obs.field" entry +// +desc = "NA"; + +// +// Output observation type to be written +// +obtype = "${OBTYPE}"; + +//////////////////////////////////////////////////////////////////////////////// + +// +// Verification grid +// +regrid = { + to_grid = ${REGRID_TO_GRID}; + method = BUDGET; + width = 2; + vld_thresh = 0.5; + shape = SQUARE; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// May be set separately in each "field" entry +// +censor_thresh = []; +censor_val = []; +cat_thresh = []; +nc_var_str = ""; + +//////////////////////////////////////////////////////////////////////////////// + +// +// Ensemble product fields to be processed +// +ens = { + ens_thresh = ${ENS_THRESH}; + vld_thresh = 1.0; + + field = [ ${ENS_FIELD} ]; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Forecast and observation fields to be verified +// +fcst = { + field = [ ${FCST_FIELD} ]; +} + +obs = { + field = [ ${OBS_FIELD} ]; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Point observation filtering options +// May be set separately in each "obs.field" entry +// +message_type = []; +sid_exc = []; +obs_thresh = [ NA ]; +obs_quality = []; +duplicate_flag = UNIQUE; +obs_summary = NONE; +obs_perc_value = 50; +skip_const = TRUE; + +// +// Observation error options +// Set dist_type to NONE to use the observation error table instead +// May be set separately in each "obs.field" entry +// +obs_error = { + flag = FALSE; // TRUE or FALSE + dist_type = NONE; // Distribution type + dist_parm = []; // Distribution parameters + inst_bias_scale = 1.0; // Instrument bias scale adjustment + inst_bias_offset = 0.0; // Instrument bias offset adjustment + min = NA; // Valid range of data + max = NA; +} + +// +// Mapping of message type group name to comma-separated list of values. +// +message_type_group_map = [ + { key = "SURFACE"; val = "ADPSFC,SFCSHP,MSONET"; }, + { key = "ANYAIR"; val = "AIRCAR,AIRCFT"; }, + { key = "ANYSFC"; val = "ADPSFC,SFCSHP,ADPUPA,PROFLR,MSONET"; }, + { key = "ONLYSF"; val = "ADPSFC,SFCSHP"; } +]; + +// +// Ensemble bin sizes +// May be set separately in each "obs.field" entry +// +ens_ssvar_bin_size = 1.0; +ens_phist_bin_size = 0.05; + +// +// Categorical thresholds to define ensemble probabilities +// May be set separately in each "fcst.field" entry +// +prob_cat_thresh = []; + + +//////////////////////////////////////////////////////////////////////////////// + +// +// Climatology data +// +climo_mean = { + + file_name = []; + field = []; + + regrid = { + method = NEAREST; + width = 1; + vld_thresh = 0.5; + shape = SQUARE; + } + + time_interp_method = DW_MEAN; + day_interval = 31; + hour_interval = 6; +} + +climo_stdev = climo_mean; +climo_stdev = { + file_name = []; +} + +// +// May be set separately in each "obs.field" entry +// +climo_cdf = { + cdf_bins = 1; + center_bins = FALSE; + write_bins = TRUE; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Point observation time window +// +obs_window = { + beg = ${OBS_WINDOW_BEGIN}; + end = ${OBS_WINDOW_END}; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Verification masking regions +// +mask = { + ${METPLUS_MASK_GRID} + ${METPLUS_MASK_POLY} + sid = []; + llpnt = []; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Confidence interval settings +// +ci_alpha = [ 0.01 ]; + +//////////////////////////////////////////////////////////////////////////////// + +// +// Interpolation methods +// +interp = { + field = BOTH; + vld_thresh = 1.0; + type = [ { method = NEAREST; width = 1; } ]; + shape = SQUARE; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Statistical output types +// +output_flag = { + ecnt = NONE; + rps = NONE; + rhist = STAT; + phist = STAT; + orank = NONE; + ssvar = STAT; + relp = STAT; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Ensemble product output types +// +ensemble_flag = { + latlon = TRUE; + mean = TRUE; + stdev = TRUE; + minus = FALSE; + plus = FALSE; + min = FALSE; + max = FALSE; + range = FALSE; + vld_count = TRUE; + frequency = TRUE; + nep = FALSE; + nmep = FALSE; + rank = FALSE; + weight = FALSE; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Random number generator +// +rng = { + type = "mt19937"; + seed = ""; +} + +//////////////////////////////////////////////////////////////////////////////// + +grid_weight_flag = NONE; +output_prefix = "${OUTPUT_PREFIX}"; +version = "V10.0.0"; + +//////////////////////////////////////////////////////////////////////////////// diff --git a/ush/templates/parm/met/EnsembleStatConfig_REFC b/ush/templates/parm/met/EnsembleStatConfig_REFC new file mode 100755 index 000000000..5d44356b6 --- /dev/null +++ b/ush/templates/parm/met/EnsembleStatConfig_REFC @@ -0,0 +1,265 @@ +//////////////////////////////////////////////////////////////////////////////// +// +// Ensemble-Stat configuration file. +// +// For additional information, see the MET_BASE/config/README file. +// +//////////////////////////////////////////////////////////////////////////////// + +// +// Output model name to be written +// +model = "${MODEL}"; + +// +// Output description to be written +// May be set separately in each "obs.field" entry +// +desc = "NA"; + +// +// Output observation type to be written +// +obtype = "${OBTYPE}"; + +//////////////////////////////////////////////////////////////////////////////// + +// +// Verification grid +// +regrid = { + to_grid = ${REGRID_TO_GRID}; + method = BUDGET; + width = 2; + vld_thresh = 0.5; + shape = SQUARE; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// May be set separately in each "field" entry +// +censor_thresh = []; +censor_val = []; +cat_thresh = []; +nc_var_str = ""; + +//////////////////////////////////////////////////////////////////////////////// + +// +// Ensemble product fields to be processed +// + +M_to_KFT(x) = x * 3.28084 * 0.001; +KM_to_KFT(x) = x * 3280.84 * 0.001; + +ens = { + ens_thresh = ${ENS_THRESH}; + vld_thresh = 1.0; + + field = [ ${ENS_FIELD} ]; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Forecast and observation fields to be verified +// + +M_to_KFT(x) = x * 3.28084 * 0.001; +KM_to_KFT(x) = x * 3280.84 * 0.001; + +fcst = { + field = [ ${FCST_FIELD} ]; +} + +obs = { + field = [ ${OBS_FIELD} ]; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Point observation filtering options +// May be set separately in each "obs.field" entry +// +message_type = []; +sid_exc = []; +obs_thresh = [ NA ]; +obs_quality = []; +duplicate_flag = UNIQUE; +obs_summary = NONE; +obs_perc_value = 50; +skip_const = TRUE; + +// +// Observation error options +// Set dist_type to NONE to use the observation error table instead +// May be set separately in each "obs.field" entry +// +obs_error = { + flag = FALSE; // TRUE or FALSE + dist_type = NONE; // Distribution type + dist_parm = []; // Distribution parameters + inst_bias_scale = 1.0; // Instrument bias scale adjustment + inst_bias_offset = 0.0; // Instrument bias offset adjustment + min = NA; // Valid range of data + max = NA; +} + +// +// Mapping of message type group name to comma-separated list of values. +// +message_type_group_map = [ + { key = "SURFACE"; val = "ADPSFC,SFCSHP,MSONET"; }, + { key = "ANYAIR"; val = "AIRCAR,AIRCFT"; }, + { key = "ANYSFC"; val = "ADPSFC,SFCSHP,ADPUPA,PROFLR,MSONET"; }, + { key = "ONLYSF"; val = "ADPSFC,SFCSHP"; } +]; + +// +// Ensemble bin sizes +// May be set separately in each "obs.field" entry +// +ens_ssvar_bin_size = 1.0; +ens_phist_bin_size = 0.05; + +// +// Categorical thresholds to define ensemble probabilities +// May be set separately in each "fcst.field" entry +// +prob_cat_thresh = []; + + +//////////////////////////////////////////////////////////////////////////////// + +// +// Climatology data +// +climo_mean = { + + file_name = []; + field = []; + + regrid = { + method = NEAREST; + width = 1; + vld_thresh = 0.5; + shape = SQUARE; + } + + time_interp_method = DW_MEAN; + day_interval = 31; + hour_interval = 6; +} + +climo_stdev = climo_mean; +climo_stdev = { + file_name = []; +} + +// +// May be set separately in each "obs.field" entry +// +climo_cdf = { + cdf_bins = 1; + center_bins = FALSE; + write_bins = TRUE; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Point observation time window +// +obs_window = { + beg = ${OBS_WINDOW_BEGIN}; + end = ${OBS_WINDOW_END}; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Verification masking regions +// +mask = { + ${METPLUS_MASK_GRID} + ${METPLUS_MASK_POLY} + sid = []; + llpnt = []; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Confidence interval settings +// +ci_alpha = [ 0.01 ]; + +//////////////////////////////////////////////////////////////////////////////// + +// +// Interpolation methods +// +interp = { + field = BOTH; + vld_thresh = 1.0; + type = [ { method = NEAREST; width = 1; } ]; + shape = SQUARE; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Statistical output types +// +output_flag = { + ecnt = NONE; + rps = NONE; + rhist = STAT; + phist = STAT; + orank = NONE; + ssvar = STAT; + relp = STAT; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Ensemble product output types +// +ensemble_flag = { + latlon = TRUE; + mean = TRUE; + stdev = TRUE; + minus = FALSE; + plus = FALSE; + min = FALSE; + max = FALSE; + range = FALSE; + vld_count = TRUE; + frequency = TRUE; + nep = FALSE; + nmep = FALSE; + rank = FALSE; + weight = FALSE; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Random number generator +// +rng = { + type = "mt19937"; + seed = ""; +} + +//////////////////////////////////////////////////////////////////////////////// + +grid_weight_flag = NONE; +output_prefix = "${OUTPUT_PREFIX}"; +version = "V10.0.0"; + +//////////////////////////////////////////////////////////////////////////////// diff --git a/ush/templates/parm/met/EnsembleStatConfig_point b/ush/templates/parm/met/EnsembleStatConfig_point new file mode 100644 index 000000000..648893e20 --- /dev/null +++ b/ush/templates/parm/met/EnsembleStatConfig_point @@ -0,0 +1,290 @@ +//////////////////////////////////////////////////////////////////////////////// +// +// Ensemble-Stat configuration file. +// +// For additional information, see the MET_BASE/config/README file. +// +//////////////////////////////////////////////////////////////////////////////// + +// +// Output model name to be written +// +model = "${MODEL}"; + +// +// Output description to be written +// May be set separately in each "obs.field" entry +// +desc = "NA"; + +// +// Output observation type to be written +// +obtype = "${OBTYPE}"; + +//////////////////////////////////////////////////////////////////////////////// + +// +// Verification grid +// May be set separately in each "field" entry +// +regrid = { + to_grid = ${REGRID_TO_GRID}; + method = BILIN; + width = 2; + vld_thresh = 0.5; + shape = SQUARE; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// May be set separately in each "field" entry +// +censor_thresh = []; +censor_val = []; +cat_thresh = []; +nc_var_str = ""; + +//////////////////////////////////////////////////////////////////////////////// + +// +// Ensemble product fields to be processed +// +ens = { + ens_thresh = 0.05; + vld_thresh = 1.0; + + field = [ ${ENS_FIELD} ]; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Neighborhood ensemble probabilities +// +nbrhd_prob = { + width = [ 5 ]; + shape = CIRCLE; + vld_thresh = 0.0; +} + +// +// NMEP smoothing methods +// +nmep_smooth = { + vld_thresh = 0.0; + shape = CIRCLE; + gaussian_dx = 81.27; + gaussian_radius = 120; + type = [ + { + method = GAUSSIAN; + width = 1; + } + ]; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Forecast and observation fields to be verified +// +fcst = { + field = [ ${FCST_FIELD} ]; +} +obs = { + field = [ ${OBS_FIELD} ]; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Point observation filtering options +// May be set separately in each "obs.field" entry +// +${METPLUS_MESSAGE_TYPE} +sid_inc = []; +sid_exc = []; +obs_thresh = [ NA ]; +obs_quality = []; +duplicate_flag = NONE; +obs_summary = NONE; +obs_perc_value = 50; +skip_const = FALSE; + +// +// Observation error options +// Set dist_type to NONE to use the observation error table instead +// May be set separately in each "obs.field" entry +// +obs_error = { + flag = FALSE; // TRUE or FALSE + dist_type = NONE; // Distribution type + dist_parm = []; // Distribution parameters + inst_bias_scale = 1.0; // Instrument bias scale adjustment + inst_bias_offset = 0.0; // Instrument bias offset adjustment + min = NA; // Valid range of data + max = NA; +} + +// +// Mapping of message type group name to comma-separated list of values +// +message_type_group_map = [ + { key = "SURFACE"; val = "ADPSFC,SFCSHP,MSONET"; }, + { key = "ANYAIR"; val = "AIRCAR,AIRCFT"; }, + { key = "ANYSFC"; val = "ADPSFC,SFCSHP,ADPUPA,PROFLR,MSONET"; }, + { key = "ONLYSF"; val = "ADPSFC,SFCSHP"; } +]; + +// +// Ensemble bin sizes +// May be set separately in each "obs.field" entry +// +ens_ssvar_bin_size = 1.0; +ens_phist_bin_size = 0.05; + +// +// Categorical thresholds to define ensemble probabilities +// May be set separately in each "fcst.field" entry +// +prob_cat_thresh = []; + +//////////////////////////////////////////////////////////////////////////////// + +// +// Climatology data +// +climo_mean = { + + file_name = []; + field = []; + + regrid = { + method = NEAREST; + width = 1; + vld_thresh = 0.5; + shape = SQUARE; + } + + time_interp_method = DW_MEAN; + day_interval = 31; + hour_interval = 6; +} + +climo_stdev = climo_mean; +climo_stdev = { + file_name = []; +} + +// +// May be set separately in each "obs.field" entry +// +climo_cdf = { + cdf_bins = 1; + center_bins = FALSE; + write_bins = TRUE; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Point observation time window +// +obs_window = { + beg = ${OBS_WINDOW_BEGIN}; + end = ${OBS_WINDOW_END}; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Verification masking regions +// +mask = { + ${METPLUS_MASK_GRID} + ${METPLUS_MASK_POLY} + sid = []; + llpnt = []; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Confidence interval settings +// +ci_alpha = [ 0.05 ]; + +//////////////////////////////////////////////////////////////////////////////// + +// +// Interpolation methods +// +interp = { + field = BOTH; + vld_thresh = 1.0; + shape = SQUARE; + + type = [ + { + method = NEAREST; + width = 1; + } + ]; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Statistical output types +// +output_flag = { + ecnt = STAT; + rps = NONE; + rhist = STAT; + phist = STAT; + orank = STAT; + ssvar = STAT; + relp = STAT; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Ensemble product output types +// +ensemble_flag = { + latlon = TRUE; + mean = TRUE; + stdev = TRUE; + minus = FALSE; + plus = FALSE; + min = FALSE; + max = FALSE; + range = TRUE; + vld_count = TRUE; + frequency = TRUE; + nep = FALSE; + nmep = FALSE; + rank = TRUE; + weight = FALSE; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Random number generator +// +rng = { + type = "mt19937"; + seed = ""; +} + +//////////////////////////////////////////////////////////////////////////////// + +grid_weight_flag = NONE; +output_prefix = "${OUTPUT_PREFIX}"; +version = "V10.0.0"; + +//////////////////////////////////////////////////////////////////////////////// diff --git a/ush/templates/parm/met/GridStatConfig_APCP_mean b/ush/templates/parm/met/GridStatConfig_APCP_mean new file mode 100755 index 000000000..6aa4c8cc3 --- /dev/null +++ b/ush/templates/parm/met/GridStatConfig_APCP_mean @@ -0,0 +1,178 @@ +//////////////////////////////////////////////////////////////////////////////// +// +// Grid-Stat configuration file. +// +// For additional information, see the MET_BASE/config/README file. +// +//////////////////////////////////////////////////////////////////////////////// + +// +// Output model name to be written +// +model = "${MODEL}"; + +// +// Output description to be written +// May be set separately in each "obs.field" entry +// +desc = "NA"; + +// +// Output observation type to be written +// +obtype = "${OBTYPE}"; + +//////////////////////////////////////////////////////////////////////////////// + +// +// Verification grid +// +regrid = { + to_grid = ${REGRID_TO_GRID}; + vld_thresh = 0.5; + method = BUDGET; + width = 2; +} + +//////////////////////////////////////////////////////////////////////////////// + +cat_thresh = [ NA ]; +cnt_thresh = [ NA ]; +cnt_logic = UNION; +wind_thresh = [ NA ]; +wind_logic = UNION; + +// +// Forecast and observation fields to be verified +// +fcst = { + field = [ ${FCST_FIELD} ]; +} + +obs = { + field = [ ${OBS_FIELD} ]; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Climatology mean data +// +climo_mean = { + + file_name = []; + field = []; + + regrid = { + vld_thresh = 0.5; + method = NEAREST; + width = 1; + } + + time_interp_method = DW_MEAN; + match_day = FALSE; + time_step = 21600; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Verification masking regions +// +mask = { + grid = []; + poly = [ ${VERIF_MASK} ]; + sid = []; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Confidence interval settings +// +ci_alpha = [ 0.05 ]; + +boot = { + interval = PCTILE; + rep_prop = 1.0; + n_rep = 0; + rng = "mt19937"; + seed = ""; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Data smoothing methods +// +interp = { + field = BOTH; + vld_thresh = 1.0; + + type = [ + { + method = NEAREST; + width = 1; + } + ]; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Neighborhood methods +// +nbrhd = { + field = BOTH; + shape = ${NEIGHBORHOOD_SHAPE}; + width = [ ${NEIGHBORHOOD_WIDTH} ]; + cov_thresh = [ >=0.5 ]; + vld_thresh = 1.0; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Statistical output types +// +output_flag = { + fho = STAT; + ctc = STAT; + cts = STAT; + mctc = NONE; + mcts = NONE; + cnt = STAT; + sl1l2 = NONE; + sal1l2 = NONE; + vl1l2 = NONE; + val1l2 = NONE; + pct = NONE; + pstd = NONE; + pjc = NONE; + prc = NONE; + nbrctc = STAT; + nbrcts = STAT; + nbrcnt = STAT; +} + +// +// NetCDF matched pairs output file +// +nc_pairs_flag = { + latlon = FALSE; + raw = FALSE; + diff = FALSE; + climo = FALSE; + weight = FALSE; + nbrhd = FALSE; + apply_mask = FALSE; +} + +//////////////////////////////////////////////////////////////////////////////// + +rank_corr_flag = FALSE; +tmp_dir = "/tmp"; +output_prefix = "${OUTPUT_PREFIX}"; +version = "V10.0.0"; + +//////////////////////////////////////////////////////////////////////////////// diff --git a/ush/templates/parm/met/GridStatConfig_APCP_prob b/ush/templates/parm/met/GridStatConfig_APCP_prob new file mode 100755 index 000000000..d6ef8cb0e --- /dev/null +++ b/ush/templates/parm/met/GridStatConfig_APCP_prob @@ -0,0 +1,178 @@ +//////////////////////////////////////////////////////////////////////////////// +// +// Grid-Stat configuration file. +// +// For additional information, see the MET_BASE/config/README file. +// +//////////////////////////////////////////////////////////////////////////////// + +// +// Output model name to be written +// +model = "${MODEL}"; + +// +// Output description to be written +// May be set separately in each "obs.field" entry +// +desc = "NA"; + +// +// Output observation type to be written +// +obtype = "${OBTYPE}"; + +//////////////////////////////////////////////////////////////////////////////// + +// +// Verification grid +// +regrid = { + to_grid = ${REGRID_TO_GRID}; + vld_thresh = 0.5; + method = BUDGET; + width = 2; +} + +//////////////////////////////////////////////////////////////////////////////// + +cat_thresh = [ NA ]; +cnt_thresh = [ NA ]; +cnt_logic = UNION; +wind_thresh = [ NA ]; +wind_logic = UNION; + +// +// Forecast and observation fields to be verified +// +fcst = { + field = [ ${FCST_FIELD} ]; +} + +obs = { + field = [ ${OBS_FIELD} ]; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Climatology mean data +// +climo_mean = { + + file_name = []; + field = []; + + regrid = { + vld_thresh = 0.5; + method = NEAREST; + width = 1; + } + + time_interp_method = DW_MEAN; + match_day = FALSE; + time_step = 21600; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Verification masking regions +// +mask = { + grid = []; + poly = [ ${VERIF_MASK} ]; + sid = []; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Confidence interval settings +// +ci_alpha = [ 0.05 ]; + +boot = { + interval = PCTILE; + rep_prop = 1.0; + n_rep = 0; + rng = "mt19937"; + seed = ""; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Data smoothing methods +// +interp = { + field = BOTH; + vld_thresh = 1.0; + + type = [ + { + method = NEAREST; + width = 1; + } + ]; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Neighborhood methods +// +nbrhd = { + field = BOTH; + shape = ${NEIGHBORHOOD_SHAPE}; + width = [ ${NEIGHBORHOOD_WIDTH} ]; + cov_thresh = [ >=0.5 ]; + vld_thresh = 1.0; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Statistical output types +// +output_flag = { + fho = NONE; + ctc = NONE; + cts = NONE; + mctc = NONE; + mcts = NONE; + cnt = NONE; + sl1l2 = NONE; + sal1l2 = NONE; + vl1l2 = NONE; + val1l2 = NONE; + pct = STAT; + pstd = STAT; + pjc = STAT; + prc = STAT; + nbrctc = NONE; + nbrcts = NONE; + nbrcnt = NONE; +} + +// +// NetCDF matched pairs output file +// +nc_pairs_flag = { + latlon = FALSE; + raw = FALSE; + diff = FALSE; + climo = FALSE; + weight = FALSE; + nbrhd = FALSE; + apply_mask = FALSE; +} + +//////////////////////////////////////////////////////////////////////////////// + +rank_corr_flag = FALSE; +tmp_dir = "/tmp"; +output_prefix = "${OUTPUT_PREFIX}"; +version = "V10.0.0"; + +//////////////////////////////////////////////////////////////////////////////// diff --git a/ush/templates/parm/met/GridStatConfig_REFL b/ush/templates/parm/met/GridStatConfig_REFC similarity index 100% rename from ush/templates/parm/met/GridStatConfig_REFL rename to ush/templates/parm/met/GridStatConfig_REFC diff --git a/ush/templates/parm/met/GridStatConfig_REFC_mean b/ush/templates/parm/met/GridStatConfig_REFC_mean new file mode 100644 index 000000000..ade839df7 --- /dev/null +++ b/ush/templates/parm/met/GridStatConfig_REFC_mean @@ -0,0 +1,219 @@ +//////////////////////////////////////////////////////////////////////////////// +// +// Grid-Stat configuration file. +// +// For additional information, see the MET_BASE/config/README file. +// +//////////////////////////////////////////////////////////////////////////////// + +// +// Output model name to be written +// +model = "${MODEL}"; + +// +// Output description to be written +// May be set separately in each "obs.field" entry +// +desc = "NA"; + +// +// Output observation type to be written +// +obtype = "${OBTYPE}"; + +//////////////////////////////////////////////////////////////////////////////// + +// +// Verification grid +// +regrid = { + to_grid = ${REGRID_TO_GRID}; + vld_thresh = 0.5; + method = BUDGET; + width = 2; + shape = SQUARE; +} + +//////////////////////////////////////////////////////////////////////////////// + +cat_thresh = []; +cnt_thresh = [ NA ]; +cnt_logic = UNION; +wind_thresh = [ NA ]; +wind_logic = UNION; + +// +// Forecast and observation fields to be verified +// + +M_to_KFT(x) = x * 3.28084 * 0.001; +KM_to_KFT(x) = x * 3280.84 * 0.001; + +fcst = { + field = [ ${FCST_FIELD} ]; +} + +obs = { + field = [ ${OBS_FIELD} ]; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Climatology mean data +// +climo_mean = { + + file_name = []; + field = []; + + regrid = { + vld_thresh = 0.5; + method = NEAREST; + width = 1; + } + + time_interp_method = DW_MEAN; + match_day = FALSE; + time_step = 21600; +} + +climo_stdev = climo_mean; +climo_stdev = { + file_name = []; +} + +climo_cdf_bins = 1; + +//////////////////////////////////////////////////////////////////////////////// + +// +// Verification masking regions +// +mask = { + grid = []; + poly = [ ${VERIF_MASK} ]; + sid = []; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Confidence interval settings +// +ci_alpha = [ 0.05 ]; + +boot = { + interval = PCTILE; + rep_prop = 1.0; + n_rep = 0; + rng = "mt19937"; + seed = ""; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Data smoothing methods +// +interp = { + field = NONE; + vld_thresh = 1.0; + shape = SQUARE; + + type = [ + { + method = NEAREST; + width = 1; + } + ]; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Neighborhood methods +// +nbrhd = { + field = BOTH; + shape = ${NEIGHBORHOOD_SHAPE}; + width = [ ${NEIGHBORHOOD_WIDTH} ]; + cov_thresh = [ >=0.5 ]; + vld_thresh = 1.0; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Fourier decomposition +// May be set separately in each "obs.field" entry +// +fourier = { + wave_1d_beg = []; + wave_1d_end = []; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Gradient statistics +// May be set separately in each "obs.field" entry +// +gradient = { + dx = [ 1 ]; + dy = [ 1 ]; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Statistical output types +// +output_flag = { + fho = STAT; + ctc = STAT; + cts = STAT; + mctc = NONE; + mcts = NONE; + cnt = STAT; + sl1l2 = NONE; + sal1l2 = NONE; + vl1l2 = NONE; + val1l2 = NONE; + vcnt = NONE; + pct = NONE; + pstd = NONE; + pjc = NONE; + prc = NONE; + eclv = NONE; + nbrctc = STAT; + nbrcts = STAT; + nbrcnt = STAT; + grad = NONE; +} + +// +// NetCDF matched pairs output file +// +nc_pairs_flag = { + latlon = FALSE; + raw = FALSE; + diff = FALSE; + climo = FALSE; + weight = FALSE; + nbrhd = FALSE; + fourier = FALSE; + gradient = FALSE; + apply_mask = FALSE; +} + +//////////////////////////////////////////////////////////////////////////////// + +grid_weight_flag = NONE; +rank_corr_flag = FALSE; +tmp_dir = "/tmp"; +output_prefix = "${OUTPUT_PREFIX}"; +version = "V10.0.0"; + +//////////////////////////////////////////////////////////////////////////////// diff --git a/ush/templates/parm/met/GridStatConfig_REFC_prob b/ush/templates/parm/met/GridStatConfig_REFC_prob new file mode 100644 index 000000000..61570c493 --- /dev/null +++ b/ush/templates/parm/met/GridStatConfig_REFC_prob @@ -0,0 +1,219 @@ +//////////////////////////////////////////////////////////////////////////////// +// +// Grid-Stat configuration file. +// +// For additional information, see the MET_BASE/config/README file. +// +//////////////////////////////////////////////////////////////////////////////// + +// +// Output model name to be written +// +model = "${MODEL}"; + +// +// Output description to be written +// May be set separately in each "obs.field" entry +// +desc = "NA"; + +// +// Output observation type to be written +// +obtype = "${OBTYPE}"; + +//////////////////////////////////////////////////////////////////////////////// + +// +// Verification grid +// +regrid = { + to_grid = ${REGRID_TO_GRID}; + vld_thresh = 0.5; + method = BUDGET; + width = 2; + shape = SQUARE; +} + +//////////////////////////////////////////////////////////////////////////////// + +cat_thresh = []; +cnt_thresh = [ NA ]; +cnt_logic = UNION; +wind_thresh = [ NA ]; +wind_logic = UNION; + +// +// Forecast and observation fields to be verified +// + +M_to_KFT(x) = x * 3.28084 * 0.001; +KM_to_KFT(x) = x * 3280.84 * 0.001; + +fcst = { + field = [ ${FCST_FIELD} ]; +} + +obs = { + field = [ ${OBS_FIELD} ]; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Climatology mean data +// +climo_mean = { + + file_name = []; + field = []; + + regrid = { + vld_thresh = 0.5; + method = NEAREST; + width = 1; + } + + time_interp_method = DW_MEAN; + match_day = FALSE; + time_step = 21600; +} + +climo_stdev = climo_mean; +climo_stdev = { + file_name = []; +} + +climo_cdf_bins = 1; + +//////////////////////////////////////////////////////////////////////////////// + +// +// Verification masking regions +// +mask = { + grid = []; + poly = [ ${VERIF_MASK} ]; + sid = []; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Confidence interval settings +// +ci_alpha = [ 0.05 ]; + +boot = { + interval = PCTILE; + rep_prop = 1.0; + n_rep = 0; + rng = "mt19937"; + seed = ""; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Data smoothing methods +// +interp = { + field = NONE; + vld_thresh = 1.0; + shape = SQUARE; + + type = [ + { + method = NEAREST; + width = 1; + } + ]; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Neighborhood methods +// +nbrhd = { + field = BOTH; + shape = ${NEIGHBORHOOD_SHAPE}; + width = [ ${NEIGHBORHOOD_WIDTH} ]; + cov_thresh = [ >=0.5 ]; + vld_thresh = 1.0; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Fourier decomposition +// May be set separately in each "obs.field" entry +// +fourier = { + wave_1d_beg = []; + wave_1d_end = []; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Gradient statistics +// May be set separately in each "obs.field" entry +// +gradient = { + dx = [ 1 ]; + dy = [ 1 ]; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Statistical output types +// +output_flag = { + fho = NONE; + ctc = NONE; + cts = NONE; + mctc = NONE; + mcts = NONE; + cnt = NONE; + sl1l2 = NONE; + sal1l2 = NONE; + vl1l2 = NONE; + val1l2 = NONE; + vcnt = NONE; + pct = STAT; + pstd = STAT; + pjc = STAT; + prc = STAT; + eclv = NONE; + nbrctc = NONE; + nbrcts = NONE; + nbrcnt = NONE; + grad = NONE; +} + +// +// NetCDF matched pairs output file +// +nc_pairs_flag = { + latlon = FALSE; + raw = FALSE; + diff = FALSE; + climo = FALSE; + weight = FALSE; + nbrhd = FALSE; + fourier = FALSE; + gradient = FALSE; + apply_mask = FALSE; +} + +//////////////////////////////////////////////////////////////////////////////// + +grid_weight_flag = NONE; +rank_corr_flag = FALSE; +tmp_dir = "/tmp"; +output_prefix = "${OUTPUT_PREFIX}"; +version = "V10.0.0"; + +//////////////////////////////////////////////////////////////////////////////// diff --git a/ush/templates/parm/met/PointStatConfig b/ush/templates/parm/met/PointStatConfig index 453f10c2e..384ee83a6 100644 --- a/ush/templates/parm/met/PointStatConfig +++ b/ush/templates/parm/met/PointStatConfig @@ -170,11 +170,6 @@ obs_window = { // mask = { grid = ${POINT_STAT_GRID}; -// poly = [ -// "${FIXverif_global}/vx_mask_files/grid2obs/CONUS.nc", -// "${FIXverif_global}/vx_mask_files/grid2obs/EAST.nc", -// "${FIXverif_global}/vx_mask_files/grid2obs/WEST.nc" -// ]; poly = ${POINT_STAT_POLY}; sid = ${POINT_STAT_STATION_ID}; llpnt = []; diff --git a/ush/templates/parm/met/PointStatConfig_mean b/ush/templates/parm/met/PointStatConfig_mean new file mode 100644 index 000000000..0b935c578 --- /dev/null +++ b/ush/templates/parm/met/PointStatConfig_mean @@ -0,0 +1,254 @@ +//////////////////////////////////////////////////////////////////////////////// +// +// Point-Stat configuration file. +// +// For additional information, see the MET_BASE/config/README file. +// +//////////////////////////////////////////////////////////////////////////////// + +// +// Output model name to be written +// +model = "${MODEL}"; + +// +// Output description to be written +// May be set separately in each "obs.field" entry +// +desc = "NA"; + +//////////////////////////////////////////////////////////////////////////////// + +// +// Verification grid +// May be set separately in each "field" entry +// +regrid = { + to_grid = ${REGRID_TO_GRID}; + method = BILIN; + width = 2; + vld_thresh = 0.5; + shape = SQUARE; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// May be set separately in each "field" entry +// +censor_thresh = []; +censor_val = []; +cat_thresh = [ NA ]; +cnt_thresh = [ NA ]; +cnt_logic = UNION; +wind_thresh = [ NA ]; +wind_logic = UNION; +eclv_points = 0.05; +rank_corr_flag = FALSE; + +// +// Forecast and observation fields to be verified +// +fcst = { + field = [ ${FCST_FIELD} ]; + }; + +obs = { + field = [ ${OBS_FIELD} ]; + }; + +//////////////////////////////////////////////////////////////////////////////// + +// +// Point observation filtering options +// May be set separately in each "obs.field" entry +// +message_type = ${POINT_STAT_MESSAGE_TYPE}; +sid_exc = []; +obs_quality = []; +duplicate_flag = NONE; +obs_summary = NONE; +obs_perc_value = 50; + +// +// Mapping of message type group name to comma-separated list of values. +// +message_type_group_map = [ + { key = "SURFACE"; val = "ADPSFC,SFCSHP,MSONET"; }, + { key = "ANYAIR"; val = "AIRCAR,AIRCFT"; }, + { key = "ANYSFC"; val = "ADPSFC,SFCSHP,ADPUPA,PROFLR,MSONET"; }, + { key = "ONLYSF"; val = "ADPSFC,SFCSHP"; }, + { key = "LANDSF"; val = "ADPSFC,MSONET"; }, + { key = "WATERSF"; val = "SFCSHP"; } +]; + +//////////////////////////////////////////////////////////////////////////////// + +// +// Climatology data +// +climo_mean = { + + file_name = []; + field = []; + + regrid = { + method = NEAREST; + width = 1; + vld_thresh = 0.5; + shape = SQUARE; + } + + time_interp_method = NEAREST; + match_month = TRUE; + match_day = TRUE; + time_step = 21600; +} + +climo_stdev = climo_mean; +climo_stdev = { + file_name = []; +} + +climo_cdf_bins = 1; +write_cdf_bins = FALSE; + +//////////////////////////////////////////////////////////////////////////////// + +// +// Land/Sea mask +// For LANDSF message types, only use forecast grid points where land = TRUE. +// For WATERSF message types, only use forecast grid points where land = FALSE. +// land_mask.flag may be set separately in each "obs.field" entry. +// +land_mask = { + flag = FALSE; + file_name = []; + field = { name = "LAND"; level = "L0"; } + regrid = { method = NEAREST; width = 1; } + thresh = eq1; +} + +// +// Topography +// For SURFACE message types, only use observations where the topo - station +// elevation difference meets the use_obs_thresh threshold. +// For the observations kept, when interpolating forecast data to the +// observation location, only use forecast grid points where the topo - station +// difference meets the interp_fcst_thresh threshold. +// topo_mask.flag may be set separately in each "obs.field" entry. +// +topo_mask = { + flag = FALSE; + file_name = []; + field = { name = "TOPO"; level = "L0"; } + regrid = { method = BILIN; width = 2; } + use_obs_thresh = ge-100&&le100; + interp_fcst_thresh = ge-50&&le50; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Point observation time window +// May be set separately in each "obs.field" entry +// +obs_window = { + beg = ${OBS_WINDOW_BEGIN}; + end = ${OBS_WINDOW_END}; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Verification masking regions +// May be set separately in each "obs.field" entry +// +mask = { + grid = ${POINT_STAT_GRID}; + poly = ${POINT_STAT_POLY}; + sid = ${POINT_STAT_STATION_ID}; + llpnt = []; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Confidence interval settings +// May be set separately in each "obs.field" entry +// +ci_alpha = [ 0.05 ]; + +boot = { + interval = PCTILE; + rep_prop = 1.0; + n_rep = 0; + rng = "mt19937"; + seed = ""; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Interpolation methods +// May be set separately in each "obs.field" entry +// +interp = { + vld_thresh = 1.0; + shape = SQUARE; + + type = [ + { + method = BILIN; + width = 2; + } + ]; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// HiRA verification method +// May be set separately in each "obs.field" entry +// +hira = { + flag = FALSE; + width = [ 2, 3, 4, 5 ]; + vld_thresh = 1.0; + cov_thresh = [ ==0.25 ]; + shape = SQUARE; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Statistical output types +// May be set separately in each "obs.field" entry +// +output_flag = { + fho = NONE; + ctc = NONE; + cts = NONE; + mctc = NONE; + mcts = NONE; + cnt = STAT; + sl1l2 = STAT; + sal1l2 = NONE; + vl1l2 = STAT; + val1l2 = NONE; + vcnt = STAT; + pct = NONE; + pstd = NONE; + pjc = NONE; + prc = NONE; + eclv = NONE; + mpr = NONE; +} + +//////////////////////////////////////////////////////////////////////////////// + +tmp_dir = "/tmp"; +output_prefix = "${OUTPUT_PREFIX}"; +version = "V10.0.0"; + +//////////////////////////////////////////////////////////////////////////////// diff --git a/ush/templates/parm/met/PointStatConfig_prob b/ush/templates/parm/met/PointStatConfig_prob new file mode 100644 index 000000000..f21ba1aaa --- /dev/null +++ b/ush/templates/parm/met/PointStatConfig_prob @@ -0,0 +1,254 @@ +//////////////////////////////////////////////////////////////////////////////// +// +// Point-Stat configuration file. +// +// For additional information, see the MET_BASE/config/README file. +// +//////////////////////////////////////////////////////////////////////////////// + +// +// Output model name to be written +// +model = "${MODEL}"; + +// +// Output description to be written +// May be set separately in each "obs.field" entry +// +desc = "NA"; + +//////////////////////////////////////////////////////////////////////////////// + +// +// Verification grid +// May be set separately in each "field" entry +// +regrid = { + to_grid = ${REGRID_TO_GRID}; + method = BILIN; + width = 2; + vld_thresh = 0.5; + shape = SQUARE; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// May be set separately in each "field" entry +// +censor_thresh = []; +censor_val = []; +cat_thresh = [ NA ]; +cnt_thresh = [ NA ]; +cnt_logic = UNION; +wind_thresh = [ NA ]; +wind_logic = UNION; +eclv_points = 0.05; +rank_corr_flag = FALSE; + +// +// Forecast and observation fields to be verified +// +fcst = { + field = [ ${FCST_FIELD} ]; + }; + +obs = { + field = [ ${OBS_FIELD} ]; + }; + +//////////////////////////////////////////////////////////////////////////////// + +// +// Point observation filtering options +// May be set separately in each "obs.field" entry +// +message_type = ${POINT_STAT_MESSAGE_TYPE}; +sid_exc = []; +obs_quality = []; +duplicate_flag = NONE; +obs_summary = NONE; +obs_perc_value = 50; + +// +// Mapping of message type group name to comma-separated list of values. +// +message_type_group_map = [ + { key = "SURFACE"; val = "ADPSFC,SFCSHP,MSONET"; }, + { key = "ANYAIR"; val = "AIRCAR,AIRCFT"; }, + { key = "ANYSFC"; val = "ADPSFC,SFCSHP,ADPUPA,PROFLR,MSONET"; }, + { key = "ONLYSF"; val = "ADPSFC,SFCSHP"; }, + { key = "LANDSF"; val = "ADPSFC,MSONET"; }, + { key = "WATERSF"; val = "SFCSHP"; } +]; + +//////////////////////////////////////////////////////////////////////////////// + +// +// Climatology data +// +climo_mean = { + + file_name = []; + field = []; + + regrid = { + method = NEAREST; + width = 1; + vld_thresh = 0.5; + shape = SQUARE; + } + + time_interp_method = NEAREST; + match_month = TRUE; + match_day = TRUE; + time_step = 21600; +} + +climo_stdev = climo_mean; +climo_stdev = { + file_name = []; +} + +climo_cdf_bins = 1; +write_cdf_bins = FALSE; + +//////////////////////////////////////////////////////////////////////////////// + +// +// Land/Sea mask +// For LANDSF message types, only use forecast grid points where land = TRUE. +// For WATERSF message types, only use forecast grid points where land = FALSE. +// land_mask.flag may be set separately in each "obs.field" entry. +// +land_mask = { + flag = FALSE; + file_name = []; + field = { name = "LAND"; level = "L0"; } + regrid = { method = NEAREST; width = 1; } + thresh = eq1; +} + +// +// Topography +// For SURFACE message types, only use observations where the topo - station +// elevation difference meets the use_obs_thresh threshold. +// For the observations kept, when interpolating forecast data to the +// observation location, only use forecast grid points where the topo - station +// difference meets the interp_fcst_thresh threshold. +// topo_mask.flag may be set separately in each "obs.field" entry. +// +topo_mask = { + flag = FALSE; + file_name = []; + field = { name = "TOPO"; level = "L0"; } + regrid = { method = BILIN; width = 2; } + use_obs_thresh = ge-100&&le100; + interp_fcst_thresh = ge-50&&le50; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Point observation time window +// May be set separately in each "obs.field" entry +// +obs_window = { + beg = ${OBS_WINDOW_BEGIN}; + end = ${OBS_WINDOW_END}; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Verification masking regions +// May be set separately in each "obs.field" entry +// +mask = { + grid = ${POINT_STAT_GRID}; + poly = ${POINT_STAT_POLY}; + sid = ${POINT_STAT_STATION_ID}; + llpnt = []; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Confidence interval settings +// May be set separately in each "obs.field" entry +// +ci_alpha = [ 0.05 ]; + +boot = { + interval = PCTILE; + rep_prop = 1.0; + n_rep = 0; + rng = "mt19937"; + seed = ""; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Interpolation methods +// May be set separately in each "obs.field" entry +// +interp = { + vld_thresh = 1.0; + shape = SQUARE; + + type = [ + { + method = BILIN; + width = 2; + } + ]; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// HiRA verification method +// May be set separately in each "obs.field" entry +// +hira = { + flag = FALSE; + width = [ 2, 3, 4, 5 ]; + vld_thresh = 1.0; + cov_thresh = [ ==0.25 ]; + shape = SQUARE; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Statistical output types +// May be set separately in each "obs.field" entry +// +output_flag = { + fho = NONE; + ctc = NONE; + cts = NONE; + mctc = NONE; + mcts = NONE; + cnt = NONE; + sl1l2 = NONE; + sal1l2 = NONE; + vl1l2 = NONE; + val1l2 = NONE; + vcnt = NONE; + pct = STAT; + pstd = STAT; + pjc = STAT; + prc = STAT; + eclv = NONE; + mpr = NONE; +} + +//////////////////////////////////////////////////////////////////////////////// + +tmp_dir = "/tmp"; +output_prefix = "${OUTPUT_PREFIX}"; +version = "V10.0.0"; + +//////////////////////////////////////////////////////////////////////////////// diff --git a/ush/templates/parm/metplus/APCP_01h.conf b/ush/templates/parm/metplus/APCP_01h.conf index 9c23ce1ee..4f3599d32 100644 --- a/ush/templates/parm/metplus/APCP_01h.conf +++ b/ush/templates/parm/metplus/APCP_01h.conf @@ -1,17 +1,19 @@ -# Grid to Grid Precipitation +# Grid to Grid Precipitation Example [dir] # Input and output data directories -INPUT_BASE = {ENV[EXPTDIR]}/{ENV[CDATE]}/postprd +INPUT_BASE = {ENV[INPUT_BASE]} FCST_GRID_STAT_INPUT_DIR = {INPUT_BASE} -OUTPUT_BASE = {ENV[EXPTDIR]} +OUTPUT_BASE = {ENV[OUTPUT_BASE]} GRID_STAT_OUTPUT_DIR = {OUTPUT_BASE} -LOG_DIR = {OUTPUT_BASE}/log +LOG_DIR = {ENV[EXPTDIR]}/log OBS_GRID_STAT_INPUT_DIR = {ENV[OBS_DIR]} +STAGING_DIR = {OUTPUT_BASE}/stage/APCP_01h + [filename_templates] # format of filenames FCST_GRID_STAT_INPUT_TEMPLATE = {ENV[NET]}.t{init?fmt=%H}z.prslevf{lead?fmt=%HHH}.tm{init?fmt=%H}.grib2 @@ -19,11 +21,17 @@ FCST_GRID_STAT_INPUT_TEMPLATE = {ENV[NET]}.t{init?fmt=%H}z.prslevf{lead?fmt=%HHH # ANLYS OBS_GRID_STAT_INPUT_TEMPLATE = {valid?fmt=%Y%m%d}/ccpa.t{valid?fmt=%H}z.01h.hrap.conus.gb2 -GRID_STAT_OUTPUT_TEMPLATE = {init?fmt=%Y%m%d%H}/metprd/grid_stat +GRID_STAT_OUTPUT_TEMPLATE = metprd/grid_stat # Location of MET poly files GRID_STAT_VERIFICATION_MASK_TEMPLATE = {MET_INSTALL_DIR}/share/met/poly/CONUS.poly +# Specify the name of the metplus.log file +LOG_METPLUS = {LOG_DIR}/metplus.log.{ENV[LOG_SUFFIX]} + +# Specify where the location and name of the final metplus_final.conf +METPLUS_CONF={OUTPUT_BASE}/metprd/grid_stat/metplus_final.APCP_01h.conf + [config] # Model to verify MODEL = {ENV[MODEL]} diff --git a/ush/templates/parm/metplus/APCP_03h.conf b/ush/templates/parm/metplus/APCP_03h.conf index d76687a7d..ce3f1f5db 100644 --- a/ush/templates/parm/metplus/APCP_03h.conf +++ b/ush/templates/parm/metplus/APCP_03h.conf @@ -1,21 +1,24 @@ -# Grid to Grid Precipitation +# Grid to Grid Precipitation Example [dir] # Input and output data directories -INPUT_BASE = {ENV[EXPTDIR]}/{ENV[CDATE]}/postprd -OUTPUT_BASE = {ENV[EXPTDIR]} +INPUT_BASE = {ENV[INPUT_BASE]} +OUTPUT_BASE = {ENV[OUTPUT_BASE]} +# Grid to Grid Precipitation FCST_PCP_COMBINE_INPUT_DIR = {INPUT_BASE} -FCST_PCP_COMBINE_OUTPUT_DIR = {ENV[EXPTDIR]}/{ENV[CDATE]}/metprd/pcp_combine +FCST_PCP_COMBINE_OUTPUT_DIR = {OUTPUT_BASE}/metprd/pcp_combine FCST_GRID_STAT_INPUT_DIR = {FCST_PCP_COMBINE_OUTPUT_DIR} OBS_PCP_COMBINE_INPUT_DIR = {ENV[OBS_DIR]} -OBS_PCP_COMBINE_OUTPUT_DIR = {ENV[EXPTDIR]}/{ENV[CDATE]}/metprd/pcp_combine +OBS_PCP_COMBINE_OUTPUT_DIR = {ENV[EXPTDIR]}/metprd/pcp_combine OBS_GRID_STAT_INPUT_DIR = {OBS_PCP_COMBINE_OUTPUT_DIR} GRID_STAT_OUTPUT_DIR = {OUTPUT_BASE} -LOG_DIR = {OUTPUT_BASE}/log +LOG_DIR = {ENV[EXPTDIR]}/log + +STAGING_DIR = {OUTPUT_BASE}/stage/APCP_03h [filename_templates] # format of filenames @@ -28,11 +31,17 @@ OBS_PCP_COMBINE_INPUT_TEMPLATE = {valid?fmt=%Y%m%d}/ccpa.t{valid?fmt=%H}z.01h.hr OBS_PCP_COMBINE_OUTPUT_TEMPLATE = {valid?fmt=%Y%m%d}/ccpa.t{valid?fmt=%H}z.hrap.conus.gb2_a{level?fmt=%HH}h OBS_GRID_STAT_INPUT_TEMPLATE = {OBS_PCP_COMBINE_OUTPUT_TEMPLATE} -GRID_STAT_OUTPUT_TEMPLATE = {init?fmt=%Y%m%d%H}/metprd/grid_stat +GRID_STAT_OUTPUT_TEMPLATE = metprd/grid_stat # Location of MET poly files GRID_STAT_VERIFICATION_MASK_TEMPLATE = {MET_INSTALL_DIR}/share/met/poly/CONUS.poly +# Specify the name of the metplus.log file +LOG_METPLUS = {LOG_DIR}/metplus.log.{ENV[LOG_SUFFIX]} + +# Specify where the location and name of the final metplus_final.conf +METPLUS_CONF={OUTPUT_BASE}/metprd/grid_stat/metplus_final.APCP_03h.conf + [config] # Model to verify MODEL = {ENV[MODEL]} diff --git a/ush/templates/parm/metplus/APCP_06h.conf b/ush/templates/parm/metplus/APCP_06h.conf index 4150e5eb8..6773ddba8 100644 --- a/ush/templates/parm/metplus/APCP_06h.conf +++ b/ush/templates/parm/metplus/APCP_06h.conf @@ -1,21 +1,23 @@ -# Grid to Grid Precipitation +# Grid to Grid Precipitation Example [dir] # Input and output data directories -INPUT_BASE = {ENV[EXPTDIR]}/{ENV[CDATE]}/postprd -OUTPUT_BASE = {ENV[EXPTDIR]} +INPUT_BASE = {ENV[INPUT_BASE]} +OUTPUT_BASE = {ENV[OUTPUT_BASE]} FCST_PCP_COMBINE_INPUT_DIR = {INPUT_BASE} -FCST_PCP_COMBINE_OUTPUT_DIR = {ENV[EXPTDIR]}/{ENV[CDATE]}/metprd/pcp_combine +FCST_PCP_COMBINE_OUTPUT_DIR = {OUTPUT_BASE}/metprd/pcp_combine FCST_GRID_STAT_INPUT_DIR = {FCST_PCP_COMBINE_OUTPUT_DIR} OBS_PCP_COMBINE_INPUT_DIR = {ENV[OBS_DIR]} -OBS_PCP_COMBINE_OUTPUT_DIR = {ENV[EXPTDIR]}/{ENV[CDATE]}/metprd/pcp_combine +OBS_PCP_COMBINE_OUTPUT_DIR = {ENV[EXPTDIR]}/metprd/pcp_combine OBS_GRID_STAT_INPUT_DIR = {OBS_PCP_COMBINE_OUTPUT_DIR} GRID_STAT_OUTPUT_DIR = {OUTPUT_BASE} -LOG_DIR = {OUTPUT_BASE}/log +LOG_DIR = {ENV[EXPTDIR]}/log + +STAGING_DIR = {OUTPUT_BASE}/stage/APCP_06h [filename_templates] # format of filenames @@ -28,11 +30,17 @@ OBS_PCP_COMBINE_INPUT_TEMPLATE = {valid?fmt=%Y%m%d}/ccpa.t{valid?fmt=%H}z.01h.hr OBS_PCP_COMBINE_OUTPUT_TEMPLATE = {valid?fmt=%Y%m%d}/ccpa.t{valid?fmt=%H}z.hrap.conus.gb2_a{level?fmt=%HH}h OBS_GRID_STAT_INPUT_TEMPLATE = {OBS_PCP_COMBINE_OUTPUT_TEMPLATE} -GRID_STAT_OUTPUT_TEMPLATE = {init?fmt=%Y%m%d%H}/metprd/grid_stat +GRID_STAT_OUTPUT_TEMPLATE = metprd/grid_stat # Location of MET poly files GRID_STAT_VERIFICATION_MASK_TEMPLATE = {MET_INSTALL_DIR}/share/met/poly/CONUS.poly +# Specify the name of the metplus.log file +LOG_METPLUS = {LOG_DIR}/metplus.log.{ENV[LOG_SUFFIX]} + +# Specify where the location and name of the final metplus_final.conf +METPLUS_CONF={OUTPUT_BASE}/metprd/grid_stat/metplus_final.APCP_06h.conf + [config] # Model to verify MODEL = {ENV[MODEL]} diff --git a/ush/templates/parm/metplus/APCP_24h.conf b/ush/templates/parm/metplus/APCP_24h.conf index acd3fd5a0..8b5a6bb98 100644 --- a/ush/templates/parm/metplus/APCP_24h.conf +++ b/ush/templates/parm/metplus/APCP_24h.conf @@ -1,21 +1,23 @@ -# Grid to Grid Precipitation +# Grid to Grid Precipitation Example [dir] # Input and output data directories -INPUT_BASE = {ENV[EXPTDIR]}/{ENV[CDATE]}/postprd -OUTPUT_BASE = {ENV[EXPTDIR]} +INPUT_BASE = {ENV[INPUT_BASE]} +OUTPUT_BASE = {ENV[OUTPUT_BASE]} FCST_PCP_COMBINE_INPUT_DIR = {INPUT_BASE} -FCST_PCP_COMBINE_OUTPUT_DIR = {ENV[EXPTDIR]}/{ENV[CDATE]}/metprd/pcp_combine +FCST_PCP_COMBINE_OUTPUT_DIR = {OUTPUT_BASE}/metprd/pcp_combine FCST_GRID_STAT_INPUT_DIR = {FCST_PCP_COMBINE_OUTPUT_DIR} OBS_PCP_COMBINE_INPUT_DIR = {ENV[OBS_DIR]} -OBS_PCP_COMBINE_OUTPUT_DIR = {ENV[EXPTDIR]}/{ENV[CDATE]}/metprd/pcp_combine +OBS_PCP_COMBINE_OUTPUT_DIR = {ENV[EXPTDIR]}/metprd/pcp_combine OBS_GRID_STAT_INPUT_DIR = {OBS_PCP_COMBINE_OUTPUT_DIR} GRID_STAT_OUTPUT_DIR = {OUTPUT_BASE} -LOG_DIR = {OUTPUT_BASE}/log +LOG_DIR = {ENV[EXPTDIR]}/log + +STAGING_DIR = {OUTPUT_BASE}/stage/APCP_24h [filename_templates] # format of filenames @@ -28,11 +30,17 @@ OBS_PCP_COMBINE_INPUT_TEMPLATE = {valid?fmt=%Y%m%d}/ccpa.t{valid?fmt=%H}z.01h.hr OBS_PCP_COMBINE_OUTPUT_TEMPLATE = {valid?fmt=%Y%m%d}/ccpa.t{valid?fmt=%H}z.hrap.conus.gb2_a{level?fmt=%HH}h OBS_GRID_STAT_INPUT_TEMPLATE = {OBS_PCP_COMBINE_OUTPUT_TEMPLATE} -GRID_STAT_OUTPUT_TEMPLATE = {init?fmt=%Y%m%d%H}/metprd/grid_stat +GRID_STAT_OUTPUT_TEMPLATE = metprd/grid_stat # Location of MET poly files GRID_STAT_VERIFICATION_MASK_TEMPLATE = {MET_INSTALL_DIR}/share/met/poly/CONUS.poly +# Specify the name of the metplus.log file +LOG_METPLUS = {LOG_DIR}/metplus.log.{ENV[LOG_SUFFIX]} + +# Specify where the location and name of the final metplus_final.conf +METPLUS_CONF={OUTPUT_BASE}/metprd/grid_stat/metplus_final.APCP_24h.conf + [config] # Model to verify MODEL = {ENV[MODEL]} diff --git a/ush/templates/parm/metplus/EnsembleStat_APCP01h.conf b/ush/templates/parm/metplus/EnsembleStat_APCP01h.conf new file mode 100644 index 000000000..46dd14a82 --- /dev/null +++ b/ush/templates/parm/metplus/EnsembleStat_APCP01h.conf @@ -0,0 +1,129 @@ +# Ensemble Stat Grid to Grid Precipitation Example + +[dir] +# Input and output data directories +INPUT_BASE = {ENV[EXPTDIR]}/{ENV[CDATE]} +OUTPUT_BASE = {ENV[EXPTDIR]} + +# Forecast model input directory for ensemble_stat +FCST_ENSEMBLE_STAT_INPUT_DIR = {INPUT_BASE} + +# Grid observation input dir for ensemble_stat +OBS_ENSEMBLE_STAT_GRID_INPUT_DIR = {ENV[OBS_DIR]} + +# output directory for ensemble_stat +ENSEMBLE_STAT_OUTPUT_DIR = {OUTPUT_BASE} + +LOG_DIR = {OUTPUT_BASE}/log + +STAGING_DIR = {OUTPUT_BASE}/stage/APCP_01h + +[filename_templates] +# FCST_ENSEMBLE_STAT_INPUT_TEMPLATE - comma separated list of ensemble members +# or a single line, - filename wildcard characters may be used, ? or *. +FCST_ENSEMBLE_STAT_INPUT_TEMPLATE = mem*/postprd/{ENV[NET]}.t{init?fmt=%H}z.prslevf{lead?fmt=%HHH}.tm{init?fmt=%H}.grib2 + +# Template to look for grid observations. +OBS_ENSEMBLE_STAT_GRID_INPUT_TEMPLATE = {valid?fmt=%Y%m%d}/ccpa.t{valid?fmt=%H}z.01h.hrap.conus.gb2 + +ENSEMBLE_STAT_OUTPUT_TEMPLATE = {init?fmt=%Y%m%d%H}/metprd/ensemble_stat + +# Specify the name of the metplus.log file +LOG_METPLUS = {LOG_DIR}/metplus.log.{ENV[LOG_SUFFIX]} + +# Specify where the location and name of the final metplus_final.conf +METPLUS_CONF={OUTPUT_BASE}/{ENV[CDATE]}/metprd/ensemble_stat/metplus_final.ensgrid_APCP_01h.conf + +[config] +# Configuration-related settings such as the process list, begin and end times, etc. +PROCESS_LIST = EnsembleStat + +# Looping by times: steps through each 'task' in the PROCESS_LIST for each +# defined time, and repeats until all times have been evaluated. +LOOP_ORDER = times + +# LOOP_BY: Set to INIT to loop over initialization times +LOOP_BY = INIT + +# Format of INIT_BEG and INT_END +INIT_TIME_FMT = %Y%m%d%H + +# Start time for METplus run +INIT_BEG = {ENV[CDATE]} + +# End time for METplus run +INIT_END = {ENV[CDATE]} + +# Increment between METplus runs in seconds. Must be >= 60 +INIT_INCREMENT = 3600 + +# List of forecast leads to process +LEAD_SEQ = {ENV[fhr_list]} + +# Used in the MET config file for: model, output_prefix +MODEL = {ENV[MODEL]} + +# Name to identify observation data in output +OBTYPE = CCPA + +OBS_ENSEMBLE_STAT_WINDOW_BEGIN = 0 +OBS_ENSEMBLE_STAT_WINDOW_END = 0 + +OBS_FILE_WINDOW_BEGIN = 0 +OBS_FILE_WINDOW_END = 0 + +# number of expected members for ensemble. Should correspond with the +# number of items in the list for FCST_ENSEMBLE_STAT_INPUT_TEMPLATE +ENSEMBLE_STAT_N_MEMBERS = {ENV[NUM_ENS_MEMBERS]} + +# ens.ens_thresh value in the MET config file +# threshold for ratio of valid files to expected files to allow app to run +ENSEMBLE_STAT_ENS_THRESH = 0.05 + +# Used in the MET config file for: regrid to_grid field +ENSEMBLE_STAT_REGRID_TO_GRID = FCST + +ENSEMBLE_STAT_OUTPUT_PREFIX = {MODEL}_APCP_01h_{OBTYPE} + +ENSEMBLE_STAT_CONFIG_FILE = {CONFIG_DIR}/EnsembleStatConfig_APCP + +# Verification Masking regions +# Indicate which grid and polygon masking region, if applicable +ENSEMBLE_STAT_MASK_GRID = + +# List of full path to poly masking files. NOTE: Only short lists of poly +# files work (those that fit on one line), a long list will result in an +# environment variable that is too long, resulting in an error. For long +# lists of poly masking files (i.e. all the mask files in the NCEP_mask +# directory), define these in the MET point_stat configuration file. +ENSEMBLE_STAT_MASK_POLY = {MET_INSTALL_DIR}/share/met/poly/CONUS.poly + +# ENSEMBLE_STAT_MET_OBS_ERR_TABLE is not required. +# If the variable is not defined, or the value is not set +# than the MET default is used. +ENSEMBLE_STAT_MET_OBS_ERR_TABLE = + +# Ensemble Variables and levels as specified in the ens field dictionary +# of the MET configuration file. Specify as ENS_VARn_NAME, ENS_VARn_LEVELS, +# (optional) ENS_VARn_OPTION +ENS_VAR1_NAME = APCP +ENS_VAR1_LEVELS = A01 +ENS_VAR1_THRESH = gt0.0,ge0.254,ge0.508,ge1.27,ge2.54 + +# Forecast Variables and levels as specified in the fcst field dictionary +# of the MET configuration file. Specify as FCST_VARn_NAME, FCST_VARn_LEVELS, +# (optional) FCST_VARn_OPTION +FCST_VAR1_NAME = APCP +FCST_VAR1_LEVELS = A01 + +FCST_VAR1_OPTIONS = ens_ssvar_bin_size = 50.0; ens_phist_bin_size = 0.05; + + +# Observation Variables and levels as specified in the obs field dictionary +# of the MET configuration file. Specify as OBS_VARn_NAME, OBS_VARn_LEVELS, +# (optional) OBS_VARn_OPTION +OBS_VAR1_NAME = {FCST_VAR1_NAME} +OBS_VAR1_LEVELS = {FCST_VAR1_LEVELS} + +OBS_VAR1_OPTIONS = {FCST_VAR1_OPTIONS} + diff --git a/ush/templates/parm/metplus/EnsembleStat_APCP03h.conf b/ush/templates/parm/metplus/EnsembleStat_APCP03h.conf new file mode 100644 index 000000000..cf7c95a23 --- /dev/null +++ b/ush/templates/parm/metplus/EnsembleStat_APCP03h.conf @@ -0,0 +1,172 @@ +# Ensemble Stat Grid to Grid Precipitation Example + +[dir] +# Input and output data directories +INPUT_BASE = {ENV[EXPTDIR]}/{ENV[CDATE]} +OUTPUT_BASE = {ENV[EXPTDIR]} + +FCST_PCP_COMBINE_INPUT_DIR = {INPUT_BASE} +FCST_PCP_COMBINE_OUTPUT_DIR = {INPUT_BASE} + +OBS_PCP_COMBINE_INPUT_DIR = {ENV[OBS_DIR]} +OBS_PCP_COMBINE_OUTPUT_DIR = {ENV[EXPTDIR]}/metprd/pcp_combine + +# Forecast model input directory for ensemble_stat +FCST_ENSEMBLE_STAT_INPUT_DIR = {FCST_PCP_COMBINE_OUTPUT_DIR} + +# Grid observation input dir for ensemble_stat +OBS_ENSEMBLE_STAT_GRID_INPUT_DIR = {OBS_PCP_COMBINE_OUTPUT_DIR} + +# Output directory for ensemble_stat +ENSEMBLE_STAT_OUTPUT_DIR = {OUTPUT_BASE} + +LOG_DIR = {OUTPUT_BASE}/log + +STAGING_DIR = {OUTPUT_BASE}/stage/APCP_03h + +[filename_templates] +# Need to have PCPCombine output data to individual member directories. +FCST_PCP_COMBINE_INPUT_TEMPLATE = {custom?fmt=%s}/postprd/{ENV[NET]}.t{init?fmt=%H}z.prslevf{lead?fmt=%HHH}.tm{init?fmt=%H}.grib2 +FCST_PCP_COMBINE_OUTPUT_TEMPLATE = {custom?fmt=%s}/metprd/pcp_combine/{ENV[NET]}.t{init?fmt=%H}z.prslevf{lead?fmt=%HHH}.tm{init?fmt=%H}_a03h + +OBS_PCP_COMBINE_INPUT_TEMPLATE = {OBS_PCP_COMBINE_INPUT_DIR}/{valid?fmt=%Y%m%d}/ccpa.t{valid?fmt=%H}z.01h.hrap.conus.gb2 +OBS_PCP_COMBINE_OUTPUT_TEMPLATE = {OBS_PCP_COMBINE_OUTPUT_DIR}/{valid?fmt=%Y%m%d}/ccpa.t{valid?fmt=%H}z.hrap.conus.gb2_a03h + +# FCST_ENSEMBLE_STAT_INPUT_TEMPLATE - comma separated list of ensemble members +# or a single line, - filename wildcard characters may be used, ? or *. +FCST_ENSEMBLE_STAT_INPUT_TEMPLATE = membegin_end_incr(1,{ENV[NUM_ENS_MEMBERS]},1,{ENV[NUM_PAD]})/metprd/pcp_combine/{ENV[NET]}.t{init?fmt=%H}z.prslevf{lead?fmt=%HHH}.tm{init?fmt=%H}_a03h + +# Template to look for grid observations. +OBS_ENSEMBLE_STAT_GRID_INPUT_TEMPLATE = {OBS_PCP_COMBINE_OUTPUT_TEMPLATE} + +ENSEMBLE_STAT_OUTPUT_TEMPLATE = {init?fmt=%Y%m%d%H}/metprd/ensemble_stat + +# Specify the name of the metplus.log file +LOG_METPLUS = {LOG_DIR}/metplus.log.{ENV[LOG_SUFFIX]} + +# Specify where the location and name of the final metplus_final.conf +METPLUS_CONF={OUTPUT_BASE}/{ENV[CDATE]}/metprd/ensemble_stat/metplus_final.ensgrid_APCP_03h.conf + +[config] +# Configuration-related settings such as the process list, begin and end times, etc. +PROCESS_LIST = PcpCombine, EnsembleStat + +# Looping by times: steps through each 'task' in the PROCESS_LIST for each +# defined time, and repeats until all times have been evaluated. +LOOP_ORDER = times + +PCP_COMBINE_CUSTOM_LOOP_LIST = membegin_end_incr(1,{ENV[NUM_ENS_MEMBERS]},1,{ENV[NUM_PAD]}) + +# LOOP_BY: Set to INIT to loop over initialization times +LOOP_BY = INIT + +# Format of INIT_BEG and INT_END +INIT_TIME_FMT = %Y%m%d%H + +# Start time for METplus run +INIT_BEG = {ENV[CDATE]} + +# End time for METplus run +INIT_END = {ENV[CDATE]} + +# Increment between METplus runs in seconds. Must be >= 60 +INIT_INCREMENT = 3600 + +# List of forecast leads to process +LEAD_SEQ = {ENV[fhr_list]} + +# Used in the MET config file for: model, output_prefix +MODEL = {ENV[MODEL]} +FCST_NATIVE_DATA_TYPE = GRIB + +# Name to identify observation data in output +OBTYPE = CCPA +OBS_NATIVE_DATA_TYPE = GRIB + +# Run pcp_combine on forecast/obs data? +FCST_PCP_COMBINE_RUN = True +OBS_PCP_COMBINE_RUN = True + +# Mode of pcp_combine to use (SUM, ADD, SUBTRACT) +FCST_PCP_COMBINE_METHOD = ADD +OBS_PCP_COMBINE_METHOD = ADD + +FCST_PCP_COMBINE_CONSTANT_INIT = True + +# Accumulation interval available in forecast data +FCST_PCP_COMBINE_INPUT_ACCUMS = 01 +FCST_PCP_COMBINE_OUTPUT_ACCUM = 03 + +# Accumulation interval available in observation data +OBS_PCP_COMBINE_INPUT_ACCUMS = 01 +OBS_PCP_COMBINE_OUTPUT_ACCUM = 03 + +# If 'bucket' output already exists, skip the PcpCombine step for the data +PCP_COMBINE_SKIP_IF_OUTPUT_EXISTS = True + +# Forecast data description variables +FCST_PCP_COMBINE_INPUT_DATATYPE = GRIB +FCST_IS_PROB = false + +OBS_ENSEMBLE_STAT_WINDOW_BEGIN = 0 +OBS_ENSEMBLE_STAT_WINDOW_END = 0 + +OBS_FILE_WINDOW_BEGIN = 0 +OBS_FILE_WINDOW_END = 0 + +# number of expected members for ensemble. Should correspond with the +# number of items in the list for FCST_ENSEMBLE_STAT_INPUT_TEMPLATE +ENSEMBLE_STAT_N_MEMBERS = {ENV[NUM_ENS_MEMBERS]} + +# ens.ens_thresh value in the MET config file +# threshold for ratio of valid files to expected files to allow app to run +ENSEMBLE_STAT_ENS_THRESH = 0.05 + +# Used in the MET config file for: regrid to_grid field +ENSEMBLE_STAT_REGRID_TO_GRID = FCST + +ENSEMBLE_STAT_OUTPUT_PREFIX = {MODEL}_APCP_03h_{OBTYPE} + +ENSEMBLE_STAT_CONFIG_FILE = {CONFIG_DIR}/EnsembleStatConfig_APCP + +# Verification Masking regions +# Indicate which grid and polygon masking region, if applicable +ENSEMBLE_STAT_MASK_GRID = + +# List of full path to poly masking files. NOTE: Only short lists of poly +# files work (those that fit on one line), a long list will result in an +# environment variable that is too long, resulting in an error. For long +# lists of poly masking files (i.e. all the mask files in the NCEP_mask +# directory), define these in the MET point_stat configuration file. +ENSEMBLE_STAT_MASK_POLY = {MET_INSTALL_DIR}/share/met/poly/CONUS.poly + +# ENSEMBLE_STAT_MET_OBS_ERR_TABLE is not required. +# If the variable is not defined, or the value is not set +# than the MET default is used. +ENSEMBLE_STAT_MET_OBS_ERR_TABLE = {MET_BASE}/table_files/obs_error_table.txt + +# Ensemble Variables and levels as specified in the ens field dictionary +# of the MET configuration file. Specify as ENS_VARn_NAME, ENS_VARn_LEVELS, +# (optional) ENS_VARn_OPTION +ENS_VAR1_NAME = APCP +ENS_VAR1_LEVELS = A03 +ENS_VAR1_THRESH = gt0.0,ge0.254,ge0.508,ge1.27,ge2.54,ge3.810,ge6.350 + +# Forecast Variables and levels as specified in the fcst field dictionary +# of the MET configuration file. Specify as FCST_VARn_NAME, FCST_VARn_LEVELS, +# (optional) FCST_VARn_OPTION +FCST_VAR1_NAME = APCP +FCST_VAR1_LEVELS = A03 + +# FIGURE OUT WHAT BIN SIZE SHOULD BE +FCST_VAR1_OPTIONS = ens_ssvar_bin_size = 50.0; ens_phist_bin_size = 0.05; + + +# Observation Variables and levels as specified in the obs field dictionary +# of the MET configuration file. Specify as OBS_VARn_NAME, OBS_VARn_LEVELS, +# (optional) OBS_VARn_OPTION +OBS_VAR1_NAME = {FCST_VAR1_NAME} +OBS_VAR1_LEVELS = {FCST_VAR1_LEVELS} + +OBS_VAR1_OPTIONS = {FCST_VAR1_OPTIONS} + diff --git a/ush/templates/parm/metplus/EnsembleStat_APCP06h.conf b/ush/templates/parm/metplus/EnsembleStat_APCP06h.conf new file mode 100644 index 000000000..9f8ddaafd --- /dev/null +++ b/ush/templates/parm/metplus/EnsembleStat_APCP06h.conf @@ -0,0 +1,172 @@ +# Ensemble Stat Grid to Grid Precipitation Example + +[dir] +# Input and output data directories +INPUT_BASE = {ENV[EXPTDIR]}/{ENV[CDATE]} +OUTPUT_BASE = {ENV[EXPTDIR]} + +FCST_PCP_COMBINE_INPUT_DIR = {INPUT_BASE} +FCST_PCP_COMBINE_OUTPUT_DIR = {INPUT_BASE} + +OBS_PCP_COMBINE_INPUT_DIR = {ENV[OBS_DIR]} +OBS_PCP_COMBINE_OUTPUT_DIR = {ENV[EXPTDIR]}/metprd/pcp_combine + +# Forecast model input directory for ensemble_stat +FCST_ENSEMBLE_STAT_INPUT_DIR = {FCST_PCP_COMBINE_OUTPUT_DIR} + +# Grid observation input dir for ensemble_stat +OBS_ENSEMBLE_STAT_GRID_INPUT_DIR = {OBS_PCP_COMBINE_OUTPUT_DIR} + +# Output directory for ensemble_stat +ENSEMBLE_STAT_OUTPUT_DIR = {OUTPUT_BASE} + +LOG_DIR = {OUTPUT_BASE}/log + +STAGING_DIR = {OUTPUT_BASE}/stage/APCP_06h + +[filename_templates] +# Need to have PCPCombine output data to individual member directories. +FCST_PCP_COMBINE_INPUT_TEMPLATE = {custom?fmt=%s}/postprd/{ENV[NET]}.t{init?fmt=%H}z.prslevf{lead?fmt=%HHH}.tm{init?fmt=%H}.grib2 +FCST_PCP_COMBINE_OUTPUT_TEMPLATE = {custom?fmt=%s}/metprd/pcp_combine/{ENV[NET]}.t{init?fmt=%H}z.prslevf{lead?fmt=%HHH}.tm{init?fmt=%H}_a06h + +OBS_PCP_COMBINE_INPUT_TEMPLATE = {OBS_PCP_COMBINE_INPUT_DIR}/{valid?fmt=%Y%m%d}/ccpa.t{valid?fmt=%H}z.01h.hrap.conus.gb2 +OBS_PCP_COMBINE_OUTPUT_TEMPLATE = {OBS_PCP_COMBINE_OUTPUT_DIR}/{valid?fmt=%Y%m%d}/ccpa.t{valid?fmt=%H}z.hrap.conus.gb2_a06h + +# FCST_ENSEMBLE_STAT_INPUT_TEMPLATE - comma separated list of ensemble members +# or a single line, - filename wildcard characters may be used, ? or *. +FCST_ENSEMBLE_STAT_INPUT_TEMPLATE = membegin_end_incr(1,{ENV[NUM_ENS_MEMBERS]},1,{ENV[NUM_PAD]})/metprd/pcp_combine/{ENV[NET]}.t{init?fmt=%H}z.prslevf{lead?fmt=%HHH}.tm{init?fmt=%H}_a06h + +# Template to look for grid observations. +OBS_ENSEMBLE_STAT_GRID_INPUT_TEMPLATE = {OBS_PCP_COMBINE_OUTPUT_TEMPLATE} + +ENSEMBLE_STAT_OUTPUT_TEMPLATE = {init?fmt=%Y%m%d%H}/metprd/ensemble_stat + +# Specify the name of the metplus.log file +LOG_METPLUS = {LOG_DIR}/metplus.log.{ENV[LOG_SUFFIX]} + +# Specify where the location and name of the final metplus_final.conf +METPLUS_CONF={OUTPUT_BASE}/{ENV[CDATE]}/metprd/ensemble_stat/metplus_final.ensgrid_APCP_06h.conf + +[config] +# Configuration-related settings such as the process list, begin and end times, etc. +PROCESS_LIST = PcpCombine, EnsembleStat + +# Looping by times: steps through each 'task' in the PROCESS_LIST for each +# defined time, and repeats until all times have been evaluated. +LOOP_ORDER = times + +PCP_COMBINE_CUSTOM_LOOP_LIST = membegin_end_incr(1,{ENV[NUM_ENS_MEMBERS]},1,{ENV[NUM_PAD]}) + +# LOOP_BY: Set to INIT to loop over initialization times +LOOP_BY = INIT + +# Format of INIT_BEG and INT_END +INIT_TIME_FMT = %Y%m%d%H + +# Start time for METplus run +INIT_BEG = {ENV[CDATE]} + +# End time for METplus run +INIT_END = {ENV[CDATE]} + +# Increment between METplus runs in seconds. Must be >= 60 +INIT_INCREMENT = 3600 + +# List of forecast leads to process +LEAD_SEQ = {ENV[fhr_list]} + +# Used in the MET config file for: model, output_prefix +MODEL = {ENV[MODEL]} +FCST_NATIVE_DATA_TYPE = GRIB + +# Name to identify observation data in output +OBTYPE = CCPA +OBS_NATIVE_DATA_TYPE = GRIB + +# Run pcp_combine on forecast/obs data? +FCST_PCP_COMBINE_RUN = True +OBS_PCP_COMBINE_RUN = True + +# Mode of pcp_combine to use (SUM, ADD, SUBTRACT) +FCST_PCP_COMBINE_METHOD = ADD +OBS_PCP_COMBINE_METHOD = ADD + +FCST_PCP_COMBINE_CONSTANT_INIT = True + +# Accumulation interval available in forecast data +FCST_PCP_COMBINE_INPUT_ACCUMS = 01 +FCST_PCP_COMBINE_OUTPUT_ACCUM = 06 + +# Accumulation interval available in observation data +OBS_PCP_COMBINE_INPUT_ACCUMS = 01 +OBS_PCP_COMBINE_OUTPUT_ACCUM = 06 + +# If 'bucket' output already exists, skip the PcpCombine step for the data +PCP_COMBINE_SKIP_IF_OUTPUT_EXISTS = True + +# Forecast data description variables +FCST_PCP_COMBINE_INPUT_DATATYPE = GRIB +FCST_IS_PROB = false + +OBS_ENSEMBLE_STAT_WINDOW_BEGIN = 0 +OBS_ENSEMBLE_STAT_WINDOW_END = 0 + +OBS_FILE_WINDOW_BEGIN = 0 +OBS_FILE_WINDOW_END = 0 + +# number of expected members for ensemble. Should correspond with the +# number of items in the list for FCST_ENSEMBLE_STAT_INPUT_TEMPLATE +ENSEMBLE_STAT_N_MEMBERS = {ENV[NUM_ENS_MEMBERS]} + +# ens.ens_thresh value in the MET config file +# threshold for ratio of valid files to expected files to allow app to run +ENSEMBLE_STAT_ENS_THRESH = 0.05 + +# Used in the MET config file for: regrid to_grid field +ENSEMBLE_STAT_REGRID_TO_GRID = FCST + +ENSEMBLE_STAT_OUTPUT_PREFIX = {MODEL}_APCP_06h_{OBTYPE} + +ENSEMBLE_STAT_CONFIG_FILE = {CONFIG_DIR}/EnsembleStatConfig_APCP + +# Verification Masking regions +# Indicate which grid and polygon masking region, if applicable +ENSEMBLE_STAT_MASK_GRID = + +# List of full path to poly masking files. NOTE: Only short lists of poly +# files work (those that fit on one line), a long list will result in an +# environment variable that is too long, resulting in an error. For long +# lists of poly masking files (i.e. all the mask files in the NCEP_mask +# directory), define these in the MET point_stat configuration file. +ENSEMBLE_STAT_MASK_POLY = {MET_INSTALL_DIR}/share/met/poly/CONUS.poly + +# ENSEMBLE_STAT_MET_OBS_ERR_TABLE is not required. +# If the variable is not defined, or the value is not set +# than the MET default is used. +ENSEMBLE_STAT_MET_OBS_ERR_TABLE = {MET_BASE}/table_files/obs_error_table.txt + +# Ensemble Variables and levels as specified in the ens field dictionary +# of the MET configuration file. Specify as ENS_VARn_NAME, ENS_VARn_LEVELS, +# (optional) ENS_VARn_OPTION +ENS_VAR1_NAME = APCP +ENS_VAR1_LEVELS = A06 +ENS_VAR1_THRESH = gt0.0,ge0.254,ge0.508,ge1.27,ge2.54,ge3.810,ge6.350 + +# Forecast Variables and levels as specified in the fcst field dictionary +# of the MET configuration file. Specify as FCST_VARn_NAME, FCST_VARn_LEVELS, +# (optional) FCST_VARn_OPTION +FCST_VAR1_NAME = APCP +FCST_VAR1_LEVELS = A06 + +# FIGURE OUT WHAT BIN SIZE SHOULD BE +FCST_VAR1_OPTIONS = ens_ssvar_bin_size = 50.0; ens_phist_bin_size = 0.05; + + +# Observation Variables and levels as specified in the obs field dictionary +# of the MET configuration file. Specify as OBS_VARn_NAME, OBS_VARn_LEVELS, +# (optional) OBS_VARn_OPTION +OBS_VAR1_NAME = {FCST_VAR1_NAME} +OBS_VAR1_LEVELS = {FCST_VAR1_LEVELS} + +OBS_VAR1_OPTIONS = {FCST_VAR1_OPTIONS} + diff --git a/ush/templates/parm/metplus/EnsembleStat_APCP24h.conf b/ush/templates/parm/metplus/EnsembleStat_APCP24h.conf new file mode 100644 index 000000000..4ce438fbb --- /dev/null +++ b/ush/templates/parm/metplus/EnsembleStat_APCP24h.conf @@ -0,0 +1,172 @@ +# Ensemble Stat Grid to Grid Precipitation Example + +[dir] +# Input and output data directories +INPUT_BASE = {ENV[EXPTDIR]}/{ENV[CDATE]} +OUTPUT_BASE = {ENV[EXPTDIR]} + +FCST_PCP_COMBINE_INPUT_DIR = {INPUT_BASE} +FCST_PCP_COMBINE_OUTPUT_DIR = {INPUT_BASE} + +OBS_PCP_COMBINE_INPUT_DIR = {ENV[OBS_DIR]} +OBS_PCP_COMBINE_OUTPUT_DIR = {ENV[EXPTDIR]}/metprd/pcp_combine + +# Forecast model input directory for ensemble_stat +FCST_ENSEMBLE_STAT_INPUT_DIR = {FCST_PCP_COMBINE_OUTPUT_DIR} + +# Grid observation input dir for ensemble_stat +OBS_ENSEMBLE_STAT_GRID_INPUT_DIR = {OBS_PCP_COMBINE_OUTPUT_DIR} + +# Output directory for ensemble_stat +ENSEMBLE_STAT_OUTPUT_DIR = {OUTPUT_BASE} + +LOG_DIR = {OUTPUT_BASE}/log + +STAGING_DIR = {OUTPUT_BASE}/stage/APCP_24h + +[filename_templates] +# Need to have PCPCombine output data to individual member directories. +FCST_PCP_COMBINE_INPUT_TEMPLATE = {custom?fmt=%s}/postprd/{ENV[NET]}.t{init?fmt=%H}z.prslevf{lead?fmt=%HHH}.tm{init?fmt=%H}.grib2 +FCST_PCP_COMBINE_OUTPUT_TEMPLATE = {custom?fmt=%s}/metprd/pcp_combine/{ENV[NET]}.t{init?fmt=%H}z.prslevf{lead?fmt=%HHH}.tm{init?fmt=%H}_a24h + +OBS_PCP_COMBINE_INPUT_TEMPLATE = {OBS_PCP_COMBINE_INPUT_DIR}/{valid?fmt=%Y%m%d}/ccpa.t{valid?fmt=%H}z.01h.hrap.conus.gb2 +OBS_PCP_COMBINE_OUTPUT_TEMPLATE = {OBS_PCP_COMBINE_OUTPUT_DIR}/{valid?fmt=%Y%m%d}/ccpa.t{valid?fmt=%H}z.hrap.conus.gb2_a24h + +# FCST_ENSEMBLE_STAT_INPUT_TEMPLATE - comma separated list of ensemble members +# or a single line, - filename wildcard characters may be used, ? or *. +FCST_ENSEMBLE_STAT_INPUT_TEMPLATE = membegin_end_incr(1,{ENV[NUM_ENS_MEMBERS]},1,{ENV[NUM_PAD]})/metprd/pcp_combine/{ENV[NET]}.t{init?fmt=%H}z.prslevf{lead?fmt=%HHH}.tm{init?fmt=%H}_a24h + +# Template to look for grid observations. +OBS_ENSEMBLE_STAT_GRID_INPUT_TEMPLATE = {OBS_PCP_COMBINE_OUTPUT_TEMPLATE} + +ENSEMBLE_STAT_OUTPUT_TEMPLATE = {init?fmt=%Y%m%d%H}/metprd/ensemble_stat + +# Specify the name of the metplus.log file +LOG_METPLUS = {LOG_DIR}/metplus.log.{ENV[LOG_SUFFIX]} + +# Specify where the location and name of the final metplus_final.conf +METPLUS_CONF={OUTPUT_BASE}/{ENV[CDATE]}/metprd/ensemble_stat/metplus_final.ensgrid_APCP_24h.conf + +[config] +# Configuration-related settings such as the process list, begin and end times, etc. +PROCESS_LIST = PcpCombine, EnsembleStat + +# Looping by times: steps through each 'task' in the PROCESS_LIST for each +# defined time, and repeats until all times have been evaluated. +LOOP_ORDER = times + +PCP_COMBINE_CUSTOM_LOOP_LIST = membegin_end_incr(1,{ENV[NUM_ENS_MEMBERS]},1,{ENV[NUM_PAD]}) + +# LOOP_BY: Set to INIT to loop over initialization times +LOOP_BY = INIT + +# Format of INIT_BEG and INT_END +INIT_TIME_FMT = %Y%m%d%H + +# Start time for METplus run +INIT_BEG = {ENV[CDATE]} + +# End time for METplus run +INIT_END = {ENV[CDATE]} + +# Increment between METplus runs in seconds. Must be >= 60 +INIT_INCREMENT = 3600 + +# List of forecast leads to process +LEAD_SEQ = {ENV[fhr_list]} + +# Used in the MET config file for: model, output_prefix +MODEL = {ENV[MODEL]} +FCST_NATIVE_DATA_TYPE = GRIB + +# Name to identify observation data in output +OBTYPE = CCPA +OBS_NATIVE_DATA_TYPE = GRIB + +# Run pcp_combine on forecast/obs data? +FCST_PCP_COMBINE_RUN = True +OBS_PCP_COMBINE_RUN = True + +# Mode of pcp_combine to use (SUM, ADD, SUBTRACT) +FCST_PCP_COMBINE_METHOD = ADD +OBS_PCP_COMBINE_METHOD = ADD + +FCST_PCP_COMBINE_CONSTANT_INIT = True + +# Accumulation interval available in forecast data +FCST_PCP_COMBINE_INPUT_ACCUMS = 01 +FCST_PCP_COMBINE_OUTPUT_ACCUM = 24 + +# Accumulation interval available in observation data +OBS_PCP_COMBINE_INPUT_ACCUMS = 01 +OBS_PCP_COMBINE_OUTPUT_ACCUM = 24 + +# If 'bucket' output already exists, skip the PcpCombine step for the data +PCP_COMBINE_SKIP_IF_OUTPUT_EXISTS = True + +# Forecast data description variables +FCST_PCP_COMBINE_INPUT_DATATYPE = GRIB +FCST_IS_PROB = false + +OBS_ENSEMBLE_STAT_WINDOW_BEGIN = 0 +OBS_ENSEMBLE_STAT_WINDOW_END = 0 + +OBS_FILE_WINDOW_BEGIN = 0 +OBS_FILE_WINDOW_END = 0 + +# number of expected members for ensemble. Should correspond with the +# number of items in the list for FCST_ENSEMBLE_STAT_INPUT_TEMPLATE +ENSEMBLE_STAT_N_MEMBERS = {ENV[NUM_ENS_MEMBERS]} + +# ens.ens_thresh value in the MET config file +# threshold for ratio of valid files to expected files to allow app to run +ENSEMBLE_STAT_ENS_THRESH = 0.05 + +# Used in the MET config file for: regrid to_grid field +ENSEMBLE_STAT_REGRID_TO_GRID = FCST + +ENSEMBLE_STAT_OUTPUT_PREFIX = {MODEL}_APCP_24h_{OBTYPE} + +ENSEMBLE_STAT_CONFIG_FILE = {CONFIG_DIR}/EnsembleStatConfig_APCP + +# Verification Masking regions +# Indicate which grid and polygon masking region, if applicable +ENSEMBLE_STAT_MASK_GRID = + +# List of full path to poly masking files. NOTE: Only short lists of poly +# files work (those that fit on one line), a long list will result in an +# environment variable that is too long, resulting in an error. For long +# lists of poly masking files (i.e. all the mask files in the NCEP_mask +# directory), define these in the MET point_stat configuration file. +ENSEMBLE_STAT_MASK_POLY = {MET_INSTALL_DIR}/share/met/poly/CONUS.poly + +# ENSEMBLE_STAT_MET_OBS_ERR_TABLE is not required. +# If the variable is not defined, or the value is not set +# than the MET default is used. +ENSEMBLE_STAT_MET_OBS_ERR_TABLE = {MET_BASE}/table_files/obs_error_table.txt + +# Ensemble Variables and levels as specified in the ens field dictionary +# of the MET configuration file. Specify as ENS_VARn_NAME, ENS_VARn_LEVELS, +# (optional) ENS_VARn_OPTION +ENS_VAR1_NAME = APCP +ENS_VAR1_LEVELS = A24 +ENS_VAR1_THRESH = gt0.0,ge0.254,ge0.508,ge1.27,ge2.54,ge3.810,ge6.350 + +# Forecast Variables and levels as specified in the fcst field dictionary +# of the MET configuration file. Specify as FCST_VARn_NAME, FCST_VARn_LEVELS, +# (optional) FCST_VARn_OPTION +FCST_VAR1_NAME = APCP +FCST_VAR1_LEVELS = A24 + +# FIGURE OUT WHAT BIN SIZE SHOULD BE +FCST_VAR1_OPTIONS = ens_ssvar_bin_size = 50.0; ens_phist_bin_size = 0.05; + + +# Observation Variables and levels as specified in the obs field dictionary +# of the MET configuration file. Specify as OBS_VARn_NAME, OBS_VARn_LEVELS, +# (optional) OBS_VARn_OPTION +OBS_VAR1_NAME = {FCST_VAR1_NAME} +OBS_VAR1_LEVELS = {FCST_VAR1_LEVELS} + +OBS_VAR1_OPTIONS = {FCST_VAR1_OPTIONS} + diff --git a/ush/templates/parm/metplus/EnsembleStat_REFC.conf b/ush/templates/parm/metplus/EnsembleStat_REFC.conf new file mode 100644 index 000000000..393f6037b --- /dev/null +++ b/ush/templates/parm/metplus/EnsembleStat_REFC.conf @@ -0,0 +1,119 @@ +# Ensemble Stat Grid to Grid REFC Example + +[dir] +# Input and output data directories +INPUT_BASE = {ENV[EXPTDIR]}/{ENV[CDATE]} +OUTPUT_BASE = {ENV[EXPTDIR]} + +# Forecast model input directory for ensemble_stat +FCST_ENSEMBLE_STAT_INPUT_DIR = {INPUT_BASE} + +# Grid observation input dir for ensemble_stat +OBS_ENSEMBLE_STAT_GRID_INPUT_DIR = {ENV[OBS_DIR]} + +# output directory for ensemble_stat +ENSEMBLE_STAT_OUTPUT_DIR = {OUTPUT_BASE} + +LOG_DIR = {OUTPUT_BASE}/log + +STAGING_DIR = {OUTPUT_BASE}/stage/REFC + +[filename_templates] +# FCST_ENSEMBLE_STAT_INPUT_TEMPLATE - comma separated list of ensemble members +# or a single line, - filename wildcard characters may be used, ? or *. +FCST_ENSEMBLE_STAT_INPUT_TEMPLATE = mem*/postprd/{ENV[NET]}.t{init?fmt=%H}z.prslevf{lead?fmt=%HHH}.tm{init?fmt=%H}.grib2 + +# Template to look for grid observations. +OBS_ENSEMBLE_STAT_GRID_INPUT_TEMPLATE = {valid?fmt=%Y%m%d}/MergedReflectivityQCComposite_00.50_{valid?fmt=%Y%m%d}-{valid?fmt=%H}0000.grib2 + +ENSEMBLE_STAT_OUTPUT_TEMPLATE = {init?fmt=%Y%m%d%H}/metprd/ensemble_stat + +ENSEMBLE_STAT_VERIFICATION_MASK_TEMPLATE = {MET_INSTALL_DIR}/share/met/poly/CONUS.poly + +LOG_METPLUS = {LOG_DIR}/metplus.log.{ENV[LOG_SUFFIX]} + +METPLUS_CONF={OUTPUT_BASE}/{ENV[CDATE]}/metprd/ensemble_stat/metplus_final.ensgrid_REFC.conf + +[config] + +## Configuration-related settings such as the process list, begin and end times, etc. +PROCESS_LIST = EnsembleStat + +# Looping by times: steps through each 'task' in the PROCESS_LIST for each +# defined time, and repeats until all times have been evaluated. +LOOP_ORDER = times + +# LOOP_BY: Set to INIT to loop over initialization times +LOOP_BY = INIT + +# Format of INIT_BEG and INT_END +INIT_TIME_FMT = %Y%m%d%H + +# Start time for METplus run +INIT_BEG = {ENV[CDATE]} + +# End time for METplus run +INIT_END = {ENV[CDATE]} + +# Increment between METplus runs in seconds. Must be >= 60 +INIT_INCREMENT = 3600 + +# List of forecast leads to process +LEAD_SEQ = {ENV[fhr_list]} + +# Used in the MET config file for: model, output_prefix +MODEL = {ENV[MODEL]} + +# Name to identify observation data in output +OBTYPE = MRMS + +OBS_ENSEMBLE_STAT_WINDOW_BEGIN = 0 +OBS_ENSEMBLE_STAT_WINDOW_END = 0 + +OBS_FILE_WINDOW_BEGIN = -300 +OBS_FILE_WINDOW_END = 300 + +# number of expected members for ensemble. Should correspond with the +# number of items in the list for FCST_ENSEMBLE_STAT_INPUT_TEMPLATE +ENSEMBLE_STAT_N_MEMBERS = {ENV[NUM_ENS_MEMBERS]} + +# ens.ens_thresh value in the MET config file +# threshold for ratio of valid files to expected files to allow app to run +ENSEMBLE_STAT_ENS_THRESH = 0.05 + +# Used in the MET config file for: regrid to_grid field +ENSEMBLE_STAT_REGRID_TO_GRID = FCST + +ENSEMBLE_STAT_OUTPUT_PREFIX = {MODEL}_REFC_{OBTYPE} + +ENSEMBLE_STAT_CONFIG_FILE = {CONFIG_DIR}/EnsembleStatConfig_REFC + +# ENSEMBLE_STAT_MET_OBS_ERR_TABLE is not required. +# If the variable is not defined, or the value is not set +# than the MET default is used. +ENSEMBLE_STAT_MET_OBS_ERR_TABLE = + +# Ensemble Variables and levels as specified in the ens field dictionary +# of the MET configuration file. Specify as ENS_VARn_NAME, ENS_VARn_LEVELS, +# (optional) ENS_VARn_OPTION +ENS_VAR1_NAME = REFC +ENS_VAR1_LEVELS = L0 +ENS_VAR1_THRESH = ge20, ge30, ge40, ge50 + +# Forecast Variables and levels as specified in the fcst field dictionary +# of the MET configuration file. Specify as FCST_VARn_NAME, FCST_VARn_LEVELS, +# (optional) FCST_VARn_OPTION +FCST_VAR1_NAME = REFC +FCST_VAR1_LEVELS = L0 + +FCST_VAR1_OPTIONS = ens_ssvar_bin_size = 50.0; ens_phist_bin_size = 0.05; + + +# Observation Variables and levels as specified in the obs field dictionary +# of the MET configuration file. Specify as OBS_VARn_NAME, OBS_VARn_LEVELS, +# (optional) OBS_VARn_OPTION + +OBS_VAR1_NAME = MergedReflectivityQCComposite +OBS_VAR1_LEVELS = Z500 +OBS_VAR1_OPTIONS = censor_thresh = lt-20; censor_val = -20.0; cnt_thresh = [ >15 ]; cnt_logic = UNION; ens_ssvar_bin_size = 50.0; ens_phist_bin_size = 0.05; + diff --git a/ush/templates/parm/metplus/EnsembleStat_RETOP.conf b/ush/templates/parm/metplus/EnsembleStat_RETOP.conf new file mode 100644 index 000000000..a90a9c754 --- /dev/null +++ b/ush/templates/parm/metplus/EnsembleStat_RETOP.conf @@ -0,0 +1,118 @@ +# Ensemble Stat Grid to Grid RETOP Example + +[dir] +# Input and output data directories +INPUT_BASE = {ENV[EXPTDIR]}/{ENV[CDATE]} +OUTPUT_BASE = {ENV[EXPTDIR]} + +# Forecast model input directory for ensemble_stat +FCST_ENSEMBLE_STAT_INPUT_DIR = {INPUT_BASE} + +# Grid observation input dir for ensemble_stat +OBS_ENSEMBLE_STAT_GRID_INPUT_DIR = {ENV[OBS_DIR]} + +# output directory for ensemble_stat +ENSEMBLE_STAT_OUTPUT_DIR = {OUTPUT_BASE} + +LOG_DIR = {OUTPUT_BASE}/log + +STAGING_DIR = {OUTPUT_BASE}/stage/RETOP + +[filename_templates] +# FCST_ENSEMBLE_STAT_INPUT_TEMPLATE - comma separated list of ensemble members +# or a single line, - filename wildcard characters may be used, ? or *. +FCST_ENSEMBLE_STAT_INPUT_TEMPLATE = mem*/postprd/{ENV[NET]}.t{init?fmt=%H}z.prslevf{lead?fmt=%HHH}.tm{init?fmt=%H}.grib2 + +# Template to look for grid observations. +OBS_ENSEMBLE_STAT_GRID_INPUT_TEMPLATE = {valid?fmt=%Y%m%d}/EchoTop_18_00.50_{valid?fmt=%Y%m%d}-{valid?fmt=%H}0000.grib2 + +ENSEMBLE_STAT_OUTPUT_TEMPLATE = {init?fmt=%Y%m%d%H}/metprd/ensemble_stat + +ENSEMBLE_STAT_VERIFICATION_MASK_TEMPLATE = {MET_INSTALL_DIR}/share/met/poly/CONUS.poly + +LOG_METPLUS = {LOG_DIR}/metplus.log.{ENV[LOG_SUFFIX]} + +METPLUS_CONF={OUTPUT_BASE}/{ENV[CDATE]}/metprd/ensemble_stat/metplus_final.ensgrid_RETOP.conf + +[config] + +## Configuration-related settings such as the process list, begin and end times, etc. +PROCESS_LIST = EnsembleStat + +# Looping by times: steps through each 'task' in the PROCESS_LIST for each +# defined time, and repeats until all times have been evaluated. +LOOP_ORDER = times + +# LOOP_BY: Set to INIT to loop over initialization times +LOOP_BY = INIT + +# Format of INIT_BEG and INT_END +INIT_TIME_FMT = %Y%m%d%H + +# Start time for METplus run +INIT_BEG = {ENV[CDATE]} + +# End time for METplus run +INIT_END = {ENV[CDATE]} + +# Increment between METplus runs in seconds. Must be >= 60 +INIT_INCREMENT = 3600 + +# List of forecast leads to process +LEAD_SEQ = {ENV[fhr_list]} + +# Used in the MET config file for: model, output_prefix +MODEL = {ENV[MODEL]} + +# Name to identify observation data in output +OBTYPE = MRMS + +OBS_ENSEMBLE_STAT_WINDOW_BEGIN = 0 +OBS_ENSEMBLE_STAT_WINDOW_END = 0 + +OBS_FILE_WINDOW_BEGIN = -300 +OBS_FILE_WINDOW_END = 300 + +# number of expected members for ensemble. Should correspond with the +# number of items in the list for FCST_ENSEMBLE_STAT_INPUT_TEMPLATE +ENSEMBLE_STAT_N_MEMBERS = {ENV[NUM_ENS_MEMBERS]} + +# ens.ens_thresh value in the MET config file +# threshold for ratio of valid files to expected files to allow app to run +ENSEMBLE_STAT_ENS_THRESH = 0.05 + +# Used in the MET config file for: regrid to_grid field +ENSEMBLE_STAT_REGRID_TO_GRID = FCST + +ENSEMBLE_STAT_OUTPUT_PREFIX = {MODEL}_RETOP_{OBTYPE} + +ENSEMBLE_STAT_CONFIG_FILE = {CONFIG_DIR}/EnsembleStatConfig_REFC + +# ENSEMBLE_STAT_MET_OBS_ERR_TABLE is not required. +# If the variable is not defined, or the value is not set +# than the MET default is used. +ENSEMBLE_STAT_MET_OBS_ERR_TABLE = + +# Ensemble Variables and levels as specified in the ens field dictionary +# of the MET configuration file. Specify as ENS_VARn_NAME, ENS_VARn_LEVELS, +# (optional) ENS_VARn_OPTION +ENS_VAR1_NAME = RETOP +ENS_VAR1_LEVELS = L0 +ENS_VAR1_THRESH = ge20, ge30, ge40, ge50 +ENS_VAR1_OPTIONS = convert(x) = M_to_KFT(x); + +# Forecast Variables and levels as specified in the fcst field dictionary +# of the MET configuration file. Specify as FCST_VARn_NAME, FCST_VARn_LEVELS, +# (optional) FCST_VARn_OPTION +FCST_VAR1_NAME = RETOP +FCST_VAR1_LEVELS = L0 +FCST_VAR1_OPTIONS = convert(x) = M_to_KFT(x); ens_ssvar_bin_size = 50.0; ens_phist_bin_size = 0.05; + +# Observation Variables and levels as specified in the obs field dictionary +# of the MET configuration file. Specify as OBS_VARn_NAME, OBS_VARn_LEVELS, +# (optional) OBS_VARn_OPTION + +OBS_VAR1_NAME = EchoTop18 +OBS_VAR1_LEVELS = Z500 +OBS_VAR1_OPTIONS = censor_thresh = lt-20; censor_val = -20.0; cnt_thresh = [ >15 ]; cnt_logic = UNION; convert(x) = KM_to_KFT(x); ens_ssvar_bin_size = 50.0; ens_phist_bin_size = 0.05; + diff --git a/ush/templates/parm/metplus/EnsembleStat_conus_sfc.conf b/ush/templates/parm/metplus/EnsembleStat_conus_sfc.conf new file mode 100644 index 000000000..09d2572d7 --- /dev/null +++ b/ush/templates/parm/metplus/EnsembleStat_conus_sfc.conf @@ -0,0 +1,170 @@ +# Ensemble Stat Grid to Point Precipitation Example + +[dir] +# Input and output directories for pb2nc +PB2NC_INPUT_DIR = {ENV[OBS_DIR]} +PB2NC_OUTPUT_DIR = {ENV[EXPTDIR]}/metprd/pb2nc + +# Input directory for ensemble_stat +FCST_ENSEMBLE_STAT_INPUT_DIR = {ENV[EXPTDIR]}/{ENV[CDATE]} + +# Point observation input dir for ensemble_stat (can also set grid obs) +OBS_ENSEMBLE_STAT_POINT_INPUT_DIR = {PB2NC_OUTPUT_DIR} + +# Output directory for ensemble_stat +OUTPUT_BASE = {ENV[EXPTDIR]} +ENSEMBLE_STAT_OUTPUT_DIR = {OUTPUT_BASE} + +LOG_DIR = {ENV[EXPTDIR]}/log + +STAGING_DIR = {OUTPUT_BASE}/stage/conus_sfc + +[filename_templates] + +# Input and output templates for pb2nc +PB2NC_INPUT_TEMPLATE = prepbufr.ndas.{valid?fmt=%Y%m%d%H} +PB2NC_OUTPUT_TEMPLATE = prepbufr.ndas.{valid?fmt=%Y%m%d%H}.nc + +# Input template for EnsembleStat can also be defined using a single +# member with wildcard characters to find multiple files +FCST_ENSEMBLE_STAT_INPUT_TEMPLATE = + mem*/postprd/{ENV[NET]}.t{init?fmt=%H}z.prslevf{lead?fmt=%HHH}.tm{init?fmt=%H}.grib2 + +OBS_ENSEMBLE_STAT_POINT_INPUT_TEMPLATE = {PB2NC_OUTPUT_TEMPLATE} + +ENSEMBLE_STAT_OUTPUT_TEMPLATE = {init?fmt=%Y%m%d%H}/metprd/ensemble_stat + +# Specify the name of the metplus.log file +LOG_METPLUS = {LOG_DIR}/metplus.log.{ENV[LOG_SUFFIX]}_sfc + +# Specify where the location and name of the final metplus_final.conf +METPLUS_CONF={OUTPUT_BASE}/{ENV[CDATE]}/metprd/ensemble_stat/metplus_final.conus_surface.conf + +# Location of MET poly files +#ENSEMBLE_STAT_VERIFICATION_MASK_TEMPLATE = {MET_INSTALL_DIR}/share/met/poly/CONUS.poly + +[config] +# Configuration-related settings such as the process list, begin and end times, etc. +PROCESS_LIST = PB2NC, EnsembleStat + +# Time looping - options are INIT, VALID, RETRO, and REALTIME +LOOP_BY = INIT + +# Start time for METplus run +INIT_BEG = {ENV[CDATE]} +# End time for METplus run +INIT_END = {ENV[CDATE]} +# Increment between METplus runs in seconds. Must be > 60 +INIT_INCREMENT = 3600 +INIT_TIME_FMT = %Y%m%d%H + +# List of forecast leads to process. +LEAD_SEQ = {ENV[fhr_list]} + +# Order of loops to process data - Options are times, processes +LOOP_ORDER = times + +# The MET ensemble_stat logging level +LOG_ENSEMBLE_STAT_VERBOSITY = 3 + +# MET Configuration files for pb2nc +PB2NC_CONFIG_FILE = {CONFIG_DIR}/PB2NCConfig + +# if True, pb2nc will skip processing a file if the output already exists +# used to speed up runs and reduce redundancy +PB2NC_SKIP_IF_OUTPUT_EXISTS = True + +# Values to pass to pb2nc config file using environment variables of the same name. +PB2NC_GRID = +PB2NC_POLY = +PB2NC_STATION_ID = +PB2NC_MESSAGE_TYPE = + +# Leave empty to process all +# PB2NC_OBS_BUFR_VAR_LIST = POB, QOB, TOB, ZOB, UOB, VOB, PMO, TOCC, TDO, HOVI, D_DPT, D_WDIR, D_WIND, D_RH, D_MIXR, D_PRMSL +PB2NC_OBS_BUFR_VAR_LIST = PMO, ZOB, TOB, TDO, QOB, UOB, VOB, PWO, TOCC, D_RH, HOVI, CEILING, D_PBL, D_CAPE, MXGS, D_WIND + +# False for no time summary, True otherwise +PB2NC_TIME_SUMMARY_FLAG = False +PB2NC_TIME_SUMMARY_BEG = 000000 ;; start time of time summary in HHMMSS format +PB2NC_TIME_SUMMARY_END = 235959 ;; end time of time summary in HHMMSS format +PB2NC_TIME_SUMMARY_VAR_NAMES = PMO,TOB,TDO,UOB,VOB,PWO,TOCC +PB2NC_TIME_SUMMARY_TYPES = min, max, range, mean, stdev, median, p80 ;; a list of the statistics to summarize + +# For both PB2NC and ensemble_stat +OBS_WINDOW_BEGIN = -1799 +OBS_WINDOW_END = 1800 + +PB2NC_WINDOW_BEGIN = {OBS_WINDOW_BEGIN} +PB2NC_WINDOW_END = {OBS_WINDOW_END} + +OBS_ENSEMBLE_STAT_WINDOW_BEGIN = {OBS_WINDOW_BEGIN} +OBS_ENSEMBLE_STAT_WINDOW_END = {OBS_WINDOW_END} + +# number of expected members for ensemble. Should correspond with the +# number of items in the list for FCST_ENSEMBLE_STAT_INPUT_TEMPLATE +ENSEMBLE_STAT_N_MEMBERS = {ENV[NUM_ENS_MEMBERS]} + +# ens.ens_thresh value in the MET config file +# threshold for ratio of valid files to expected files to allow app to run +ENSEMBLE_STAT_ENS_THRESH = 0.05 + +# Used in the MET config file for: regrid to_grid field +ENSEMBLE_STAT_REGRID_TO_GRID = NONE + +ENSEMBLE_STAT_CONFIG_FILE = {CONFIG_DIR}/EnsembleStatConfig_point + +# Model/fcst and obs name, e.g. GFS, NAM, GDAS, etc. +MODEL = {ENV[MODEL]} +OBTYPE = NDAS + +# Used in the MET config file +ENSEMBLE_STAT_MESSAGE_TYPE = ADPSFC + +# Verification Masking regions +# Indicate which grid and polygon masking region, if applicable +ENSEMBLE_STAT_MASK_GRID = + +# List of full path to poly masking files. NOTE: Only short lists of poly +# files work (those that fit on one line), a long list will result in an +# environment variable that is too long, resulting in an error. For long +# lists of poly masking files (i.e. all the mask files in the NCEP_mask +# directory), define these in the MET point_stat configuration file. +ENSEMBLE_STAT_MASK_POLY = {MET_INSTALL_DIR}/share/met/poly/CONUS.poly + +# MET_OBS_ERR_TABLE is not required. +# If the variable is not defined, or the value is not set +# than the MET default is used. +ENSEMBLE_STAT_MET_OBS_ERR_TABLE = {MET_BASE}/table_files/obs_error_table.txt + +# Variables and levels as specified in the field dictionary of the MET +# configuration file. Specify as FCST_VARn_NAME, FCST_VARn_LEVELS, +# (optional) FCST_VARn_OPTION + +ENS_VAR1_NAME = TMP +ENS_VAR1_LEVELS = Z02 +ENS_VAR1_THRESH = >=293, >=298, >=303 + +ENS_VAR2_NAME = DPT +ENS_VAR2_LEVELS = Z2 +ENS_VAR2_THRESH = >=288, >=293, >=298 + +ENS_VAR3_NAME = WIND +ENS_VAR3_LEVELS = Z10 +ENS_VAR3_THRESH = >=5, >=10 +ENS_VAR3_OPTIONS = GRIB2_pdt = 0; ;; derive instantaneous 10-m wind from U/V components, overriding max 10-m wind + +BOTH_VAR1_NAME = TMP +BOTH_VAR1_LEVELS = Z2 +BOTH_VAR1_THRESH = >=293, >=298, >=303 + +BOTH_VAR2_NAME = DPT +BOTH_VAR2_LEVELS = Z2 +BOTH_VAR2_THRESH = >=288, >=293, >=298 + +BOTH_VAR3_NAME = WIND +BOTH_VAR3_LEVELS = Z10 +BOTH_VAR3_THRESH = >=5, >=10 +BOTH_VAR3_OPTIONS = GRIB2_pdt = 0; ;; derive instantaneous 10-m wind from U/V components, overriding max 10-m wind + +ENSEMBLE_STAT_OUTPUT_PREFIX = {MODEL}_ADPSFC_{OBTYPE} diff --git a/ush/templates/parm/metplus/EnsembleStat_upper_air.conf b/ush/templates/parm/metplus/EnsembleStat_upper_air.conf new file mode 100644 index 000000000..19d982ecd --- /dev/null +++ b/ush/templates/parm/metplus/EnsembleStat_upper_air.conf @@ -0,0 +1,173 @@ +# Ensemble Stat Grid to Point Precipitation Example + +[dir] +# Input and output directories for pb2nc +PB2NC_INPUT_DIR = {ENV[OBS_DIR]} +PB2NC_OUTPUT_DIR = {ENV[EXPTDIR]}/metprd/pb2nc + +# Input directory for ensemble_stat +FCST_ENSEMBLE_STAT_INPUT_DIR = {ENV[EXPTDIR]}/{ENV[CDATE]} + +# Point observation input dir for ensemble_stat (can also set grid obs) +OBS_ENSEMBLE_STAT_POINT_INPUT_DIR = {PB2NC_OUTPUT_DIR} + +# Output directory for ensemble_stat +OUTPUT_BASE = {ENV[EXPTDIR]} +ENSEMBLE_STAT_OUTPUT_DIR = {OUTPUT_BASE} + +LOG_DIR = {ENV[EXPTDIR]}/log + +STAGING_DIR = {OUTPUT_BASE}/stage/upper_air + +[filename_templates] + +# Input and output templates for pb2nc +PB2NC_INPUT_TEMPLATE = prepbufr.ndas.{valid?fmt=%Y%m%d%H} +PB2NC_OUTPUT_TEMPLATE = prepbufr.ndas.{valid?fmt=%Y%m%d%H}.nc + +# Input ensemble template - comma separated list of ensemble members +#FCST_ENSEMBLE_STAT_INPUT_TEMPLATE = +# mem01/postprd/{ENV[NET]}.t{init?fmt=%H}z.bgdawpf{lead?fmt=%HHH}.tm{init?fmt=%H}.grib2, +# mem02/postprd/{ENV[NET]}.t{init?fmt=%H}z.bgdawpf{lead?fmt=%HHH}.tm{init?fmt=%H}.grib2, + +# Input template for EnsembleStat can also be defined using a single +# member with wildcard characters to find multiple files +FCST_ENSEMBLE_STAT_INPUT_TEMPLATE = + mem*/postprd/{ENV[NET]}.t{init?fmt=%H}z.prslevf{lead?fmt=%HHH}.tm{init?fmt=%H}.grib2 + +OBS_ENSEMBLE_STAT_POINT_INPUT_TEMPLATE = {PB2NC_OUTPUT_TEMPLATE} + +ENSEMBLE_STAT_OUTPUT_TEMPLATE = {init?fmt=%Y%m%d%H}/metprd/ensemble_stat + +# Specify the name of the metplus.log file +LOG_METPLUS = {LOG_DIR}/metplus.log.{ENV[LOG_SUFFIX]}_upa + +# Specify where the location and name of the final metplus_final.conf +METPLUS_CONF={OUTPUT_BASE}/{ENV[CDATE]}/metprd/ensemble_stat/metplus_final.upper_air.conf + +# Location of MET poly files +#ENSEMBLE_STAT_VERIFICATION_MASK_TEMPLATE = {MET_INSTALL_DIR}/share/met/poly/CONUS.poly + +[config] +# Configuration-related settings such as the process list, begin and end times, etc. +PROCESS_LIST = PB2NC, EnsembleStat + +# Time looping - options are INIT, VALID, RETRO, and REALTIME +LOOP_BY = INIT + +# Start time for METplus run +INIT_BEG = {ENV[CDATE]} +# End time for METplus run +INIT_END = {ENV[CDATE]} +# Increment between METplus runs in seconds. Must be > 60 +INIT_INCREMENT = 3600 +INIT_TIME_FMT = %Y%m%d%H + +# List of forecast leads to process. +LEAD_SEQ = begin_end_incr(0,{ENV[fhr_last]},6) + +# Order of loops to process data - Options are times, processes +LOOP_ORDER = times + +# The MET ensemble_stat logging level +LOG_ENSEMBLE_STAT_VERBOSITY = 3 + +# MET Configuration files for pb2nc +PB2NC_CONFIG_FILE = {CONFIG_DIR}/PB2NCConfig + +# if True, pb2nc will skip processing a file if the output already exists +# used to speed up runs and reduce redundancy +PB2NC_SKIP_IF_OUTPUT_EXISTS = True + +# Values to pass to pb2nc config file using environment variables of the same name. +PB2NC_GRID = +PB2NC_POLY = +PB2NC_STATION_ID = +PB2NC_MESSAGE_TYPE = + +# Leave empty to process all +# PB2NC_OBS_BUFR_VAR_LIST = POB, QOB, TOB, ZOB, UOB, VOB, PMO, TOCC, TDO, HOVI, D_DPT, D_WDIR, D_WIND, D_RH, D_MIXR, D_PRMSL +PB2NC_OBS_BUFR_VAR_LIST = PMO, ZOB, TOB, TDO, QOB, UOB, VOB, PWO, TOCC, D_RH, HOVI, CEILING, D_PBL, D_CAPE, MXGS, D_WIND + +# False for no time summary, True otherwise +PB2NC_TIME_SUMMARY_FLAG = False +PB2NC_TIME_SUMMARY_BEG = 000000 ;; start time of time summary in HHMMSS format +PB2NC_TIME_SUMMARY_END = 235959 ;; end time of time summary in HHMMSS format +PB2NC_TIME_SUMMARY_VAR_NAMES = PMO,TOB,TDO,UOB,VOB,PWO,TOCC +PB2NC_TIME_SUMMARY_TYPES = min, max, range, mean, stdev, median, p80 ;; a list of the statistics to summarize + +# For both PB2NC and ensemble_stat +OBS_WINDOW_BEGIN = -1799 +OBS_WINDOW_END = 1800 + +PB2NC_WINDOW_BEGIN = {OBS_WINDOW_BEGIN} +PB2NC_WINDOW_END = {OBS_WINDOW_END} + +OBS_ENSEMBLE_STAT_WINDOW_BEGIN = {OBS_WINDOW_BEGIN} +OBS_ENSEMBLE_STAT_WINDOW_END = {OBS_WINDOW_END} + +# number of expected members for ensemble. Should correspond with the +# number of items in the list for FCST_ENSEMBLE_STAT_INPUT_TEMPLATE +ENSEMBLE_STAT_N_MEMBERS = 10 + +# ens.ens_thresh value in the MET config file +# threshold for ratio of valid files to expected files to allow app to run +ENSEMBLE_STAT_ENS_THRESH = 0.05 + +# Used in the MET config file for: regrid to_grid field +ENSEMBLE_STAT_REGRID_TO_GRID = NONE + +ENSEMBLE_STAT_CONFIG_FILE = {CONFIG_DIR}/EnsembleStatConfig_point + +# Model/fcst and obs name, e.g. GFS, NAM, GDAS, etc. +MODEL = {ENV[MODEL]} +OBTYPE = NDAS + +# Used in the MET config file +ENSEMBLE_STAT_MESSAGE_TYPE = ADPUPA + +# Verification Masking regions +# Indicate which grid and polygon masking region, if applicable +ENSEMBLE_STAT_MASK_GRID = + +# List of full path to poly masking files. NOTE: Only short lists of poly +# files work (those that fit on one line), a long list will result in an +# environment variable that is too long, resulting in an error. For long +# lists of poly masking files (i.e. all the mask files in the NCEP_mask +# directory), define these in the MET point_stat configuration file. +ENSEMBLE_STAT_MASK_POLY = {MET_INSTALL_DIR}/share/met/poly/CONUS.poly + +# MET_OBS_ERR_TABLE is not required. +# If the variable is not defined, or the value is not set +# than the MET default is used. +ENSEMBLE_STAT_MET_OBS_ERR_TABLE = {MET_BASE}/table_files/obs_error_table.txt + +# Variables and levels as specified in the field dictionary of the MET +# configuration file. Specify as FCST_VARn_NAME, FCST_VARn_LEVELS, +# (optional) FCST_VARn_OPTION + +ENS_VAR1_NAME = TMP +ENS_VAR1_LEVELS = P850 +ENS_VAR1_THRESH = >=288, >=293, >=298 + +ENS_VAR2_NAME = DPT +ENS_VAR2_LEVELS = P850 +ENS_VAR2_THRESH = >=283, >=288, >=293 + +ENS_VAR3_NAME = WIND +ENS_VAR3_LEVELS = P850 +ENS_VAR3_THRESH = >=5, >=10 + +BOTH_VAR1_NAME = TMP +BOTH_VAR1_LEVELS = P850 +BOTH_VAR1_THRESH = >=288, >=293, >=298 + +BOTH_VAR2_NAME = DPT +BOTH_VAR2_LEVELS = P850 +BOTH_VAR2_THRESH = >=283, >=288, >=293 + +BOTH_VAR3_NAME = WIND +BOTH_VAR3_LEVELS = P850 +BOTH_VAR3_THRESH = >=5, >=10 + +ENSEMBLE_STAT_OUTPUT_PREFIX = {MODEL}_ADPUPA_{OBTYPE} diff --git a/ush/templates/parm/metplus/GridStat_APCP01h_mean.conf b/ush/templates/parm/metplus/GridStat_APCP01h_mean.conf new file mode 100644 index 000000000..16cd58e7f --- /dev/null +++ b/ush/templates/parm/metplus/GridStat_APCP01h_mean.conf @@ -0,0 +1,95 @@ +# Ensemble Mean Grid to Grid Precipitation Example + +[dir] +# Input and output data directories +INPUT_BASE = {ENV[INPUT_BASE]} +OUTPUT_BASE = {ENV[EXPTDIR]} + +FCST_GRID_STAT_INPUT_DIR = {INPUT_BASE} +OBS_GRID_STAT_INPUT_DIR = {ENV[OBS_DIR]} + +GRID_STAT_OUTPUT_DIR = {OUTPUT_BASE} + +LOG_DIR = {OUTPUT_BASE}/log + +STAGING_DIR = {OUTPUT_BASE}/stage/APCP_01h_mean + +[filename_templates] +# format of filenames +FCST_GRID_STAT_INPUT_TEMPLATE = ensemble_stat_{ENV[MODEL]}_APCP_{ENV[acc]}_{OBTYPE}_{valid?fmt=%Y%m%d}_{valid?fmt=%H%M%S}V_ens.nc + +OBS_GRID_STAT_INPUT_TEMPLATE = {valid?fmt=%Y%m%d}/ccpa.t{valid?fmt=%H}z.01h.hrap.conus.gb2 + +GRID_STAT_OUTPUT_TEMPLATE = {init?fmt=%Y%m%d%H}/metprd/ensemble_stat_mean + +# Location of MET poly files +GRID_STAT_VERIFICATION_MASK_TEMPLATE = {MET_INSTALL_DIR}/share/met/poly/CONUS.poly + +# Specify the name of the metplus.log file +LOG_METPLUS = {LOG_DIR}/metplus.log.{ENV[LOG_SUFFIX]} + +# Specify where the location and name of the final metplus_final.conf +METPLUS_CONF={OUTPUT_BASE}/{ENV[CDATE]}/metprd/ensemble_stat_mean/metplus_final.APCP01.conf + +[config] +# List of applications to run +PROCESS_LIST = GridStat + +# time looping - options are INIT, VALID, RETRO, and REALTIME +LOOP_BY = INIT + +# Start time for METplus run +INIT_BEG = {ENV[CDATE]} +# End time for METplus run +INIT_END = {ENV[CDATE]} +# Increment between METplus runs in seconds. Must be > 60 +INIT_INCREMENT = 3600 +INIT_TIME_FMT = %Y%m%d%H + +# list of forecast leads to process. +# LEAD_SEQ = begin_end_incr(1,24,1) +LEAD_SEQ = {ENV[fhr_list]} + +# Options are times, processes +# times = run all items in the PROCESS_LIST for a single initialization +# time, then repeat until all times have been evaluated. +# processes = run each item in the PROCESS_LIST for all times +# specified, then repeat for the next item in the PROCESS_LIST. +LOOP_ORDER = times + +# Model to verify +MODEL = {ENV[MODEL]}_mean +FCST_NATIVE_DATA_TYPE = GRIB + +# Set obtype to vx +OBTYPE = CCPA +OBS_NATIVE_DATA_TYPE = GRIB + +# location of grid_stat MET config file +GRID_STAT_CONFIG_FILE = {CONFIG_DIR}/GridStatConfig_APCP_mean +GRID_STAT_OUTPUT_PREFIX = {ENV[MODEL]}_APCP_01h_{OBTYPE}_mean + +# run pcp_combine on forecast/obs data? +FCST_PCP_COMBINE_RUN = False +OBS_PCP_COMBINE_RUN = False + +# Set grid to verify on +GRID_STAT_REGRID_TO_GRID = FCST + +# list of variables to compare +# thresholds in mm, equal to .01",.02",.05",.10",.25",.50",.75",1.0" +FCST_VAR1_NAME = APCP_01_A01_ENS_MEAN +FCST_VAR1_LEVELS = A01 + +BOTH_VAR1_THRESH = gt0.0,ge0.254,ge0.508,ge1.27,ge2.54 + +OBS_VAR1_NAME = APCP +OBS_VAR1_LEVELS = A01 + +# Neighborhood shape and widths +GRID_STAT_NEIGHBORHOOD_SHAPE = SQUARE +GRID_STAT_NEIGHBORHOOD_WIDTH = 3,5,7 + +# Forecast data description variables +FCST_IS_PROB = False + diff --git a/ush/templates/parm/metplus/GridStat_APCP01h_prob.conf b/ush/templates/parm/metplus/GridStat_APCP01h_prob.conf new file mode 100644 index 000000000..81b3d9a6d --- /dev/null +++ b/ush/templates/parm/metplus/GridStat_APCP01h_prob.conf @@ -0,0 +1,129 @@ +# Ensemble Prob Grid to Grid Precipitation Example + +[dir] +# Input and output data directories +INPUT_BASE = {ENV[INPUT_BASE]} +OUTPUT_BASE = {ENV[EXPTDIR]} + +FCST_GRID_STAT_INPUT_DIR = {INPUT_BASE} +OBS_GRID_STAT_INPUT_DIR = {ENV[OBS_DIR]} + +GRID_STAT_OUTPUT_DIR = {OUTPUT_BASE} + +LOG_DIR = {OUTPUT_BASE}/log + +STAGING_DIR = {OUTPUT_BASE}/stage/APCP_01h_prob + +[filename_templates] +# format of filenames +FCST_GRID_STAT_INPUT_TEMPLATE = ensemble_stat_{ENV[MODEL]}_APCP_{ENV[acc]}_{OBTYPE}_{valid?fmt=%Y%m%d}_{valid?fmt=%H%M%S}V_ens.nc + +OBS_GRID_STAT_INPUT_TEMPLATE = {valid?fmt=%Y%m%d}/ccpa.t{valid?fmt=%H}z.01h.hrap.conus.gb2 + +GRID_STAT_OUTPUT_TEMPLATE = {init?fmt=%Y%m%d%H}/metprd/ensemble_stat_prob + +# Location of MET poly files +GRID_STAT_VERIFICATION_MASK_TEMPLATE = {MET_INSTALL_DIR}/share/met/poly/CONUS.poly + +# Specify the name of the metplus.log file +LOG_METPLUS = {LOG_DIR}/metplus.log.{ENV[LOG_SUFFIX]} + +# Specify where the location and name of the final metplus_final.conf +METPLUS_CONF={OUTPUT_BASE}/{ENV[CDATE]}/metprd/ensemble_stat_prob/metplus_final.APCP01.conf + +[config] +# List of applications to run +PROCESS_LIST = GridStat + +# time looping - options are INIT, VALID, RETRO, and REALTIME +LOOP_BY = INIT + +# Start time for METplus run +INIT_BEG = {ENV[CDATE]} +# End time for METplus run +INIT_END = {ENV[CDATE]} +# Increment between METplus runs in seconds. Must be > 60 +INIT_INCREMENT = 3600 +INIT_TIME_FMT = %Y%m%d%H + +# list of forecast leads to process. +# LEAD_SEQ = begin_end_incr(1,24,1) +LEAD_SEQ = {ENV[fhr_list]} + +# Options are times, processes +# times = run all items in the PROCESS_LIST for a single initialization +# time, then repeat until all times have been evaluated. +# processes = run each item in the PROCESS_LIST for all times +# specified, then repeat for the next item in the PROCESS_LIST. +LOOP_ORDER = times + +# Model to verify +MODEL = {ENV[MODEL]}_prob +FCST_NATIVE_DATA_TYPE = GRIB + +# Set obtype to vx +OBTYPE = CCPA +OBS_NATIVE_DATA_TYPE = GRIB + +# location of grid_stat MET config file +GRID_STAT_CONFIG_FILE = {CONFIG_DIR}/GridStatConfig_APCP_prob +GRID_STAT_OUTPUT_PREFIX = {ENV[MODEL]}_APCP_01h_{OBTYPE}_prob + +# run pcp_combine on forecast/obs data? +FCST_PCP_COMBINE_RUN = False +OBS_PCP_COMBINE_RUN = False + +# Set grid to verify on +GRID_STAT_REGRID_TO_GRID = FCST + +# list of variables to compare +# thresholds in mm, equal to .01",.02",.05",.10",.25",.50",.75",1.0" +# +FCST_VAR1_NAME = APCP_01_A01_ENS_FREQ_gt0.0 +FCST_VAR1_LEVELS = A01 +FCST_VAR1_THRESH = ==0.1 + +OBS_VAR1_NAME = APCP +OBS_VAR1_LEVELS = A01 +OBS_VAR1_THRESH = >0.0 + +FCST_VAR2_NAME = APCP_01_A01_ENS_FREQ_ge0.254 +FCST_VAR2_LEVELS = A01 +FCST_VAR2_THRESH = ==0.1 + +OBS_VAR2_NAME = APCP +OBS_VAR2_LEVELS = A01 +OBS_VAR2_THRESH = >=0.254 + +FCST_VAR3_NAME = APCP_01_A01_ENS_FREQ_ge0.508 +FCST_VAR3_LEVELS = A01 +FCST_VAR3_THRESH = ==0.1 + +OBS_VAR3_NAME = APCP +OBS_VAR3_LEVELS = A01 +OBS_VAR3_THRESH = >=0.508 + +FCST_VAR4_NAME = APCP_01_A01_ENS_FREQ_ge1.27 +FCST_VAR4_LEVELS = A01 +FCST_VAR4_THRESH = ==0.1 + +OBS_VAR4_NAME = APCP +OBS_VAR4_LEVELS = A01 +OBS_VAR4_THRESH = >=1.27 + +FCST_VAR5_NAME = APCP_01_A01_ENS_FREQ_ge2.54 +FCST_VAR5_LEVELS = A01 +FCST_VAR5_THRESH = ==0.1 + +OBS_VAR5_NAME = APCP +OBS_VAR5_LEVELS = A01 +OBS_VAR5_THRESH = >=2.54 + + +# Neighborhood shape and widths +GRID_STAT_NEIGHBORHOOD_SHAPE = SQUARE +GRID_STAT_NEIGHBORHOOD_WIDTH = 3,5,7 + +# Forecast data description variables +FCST_IS_PROB = True +FCST_PROB_IN_GRIB_PDS = False diff --git a/ush/templates/parm/metplus/GridStat_APCP03h_mean.conf b/ush/templates/parm/metplus/GridStat_APCP03h_mean.conf new file mode 100644 index 000000000..536ce241f --- /dev/null +++ b/ush/templates/parm/metplus/GridStat_APCP03h_mean.conf @@ -0,0 +1,89 @@ +# Ensemble Mean Grid to Grid Precipitation Example + +[dir] +# Input and output data directories +INPUT_BASE = {ENV[INPUT_BASE]} +OUTPUT_BASE = {ENV[EXPTDIR]} + +FCST_GRID_STAT_INPUT_DIR = {INPUT_BASE} +OBS_GRID_STAT_INPUT_DIR = {ENV[EXPTDIR]}/metprd/pcp_combine + +GRID_STAT_OUTPUT_DIR = {OUTPUT_BASE} + +LOG_DIR = {OUTPUT_BASE}/log + +STAGING_DIR = {OUTPUT_BASE}/stage/APCP_03h_mean + +[filename_templates] +# format of filenames +FCST_GRID_STAT_INPUT_TEMPLATE = ensemble_stat_{ENV[MODEL]}_APCP_{ENV[acc]}_{OBTYPE}_{valid?fmt=%Y%m%d}_{valid?fmt=%H%M%S}V_ens.nc + +OBS_GRID_STAT_INPUT_TEMPLATE = {OBS_GRID_STAT_INPUT_DIR}/{valid?fmt=%Y%m%d}/ccpa.t{valid?fmt=%H}z.hrap.conus.gb2_a03h + +GRID_STAT_OUTPUT_TEMPLATE = {init?fmt=%Y%m%d%H}/metprd/ensemble_stat_mean + +# Location of MET poly files +GRID_STAT_VERIFICATION_MASK_TEMPLATE = {MET_INSTALL_DIR}/share/met/poly/CONUS.poly + +# Specify the name of the metplus.log file +LOG_METPLUS = {LOG_DIR}/metplus.log.{ENV[LOG_SUFFIX]} + +# Specify where the location and name of the final metplus_final.conf +METPLUS_CONF={OUTPUT_BASE}/{ENV[CDATE]}/metprd/ensemble_stat_mean/metplus_final.APCP03.conf + +[config] +# List of applications to run +PROCESS_LIST = GridStat + +# time looping - options are INIT, VALID, RETRO, and REALTIME +LOOP_BY = INIT + +# Start time for METplus run +INIT_BEG = {ENV[CDATE]} +# End time for METplus run +INIT_END = {ENV[CDATE]} +# Increment between METplus runs in seconds. Must be > 60 +INIT_INCREMENT = 3600 +INIT_TIME_FMT = %Y%m%d%H + +# list of forecast leads to process. +# LEAD_SEQ = begin_end_incr(1,24,1) +LEAD_SEQ = {ENV[fhr_list]} + +# Options are times, processes +# times = run all items in the PROCESS_LIST for a single initialization +# time, then repeat until all times have been evaluated. +# processes = run each item in the PROCESS_LIST for all times +# specified, then repeat for the next item in the PROCESS_LIST. +LOOP_ORDER = times + +# Model to verify +MODEL = {ENV[MODEL]}_mean + +# Set obtype to vx +OBTYPE = CCPA + +# location of grid_stat MET config file +GRID_STAT_CONFIG_FILE = {CONFIG_DIR}/GridStatConfig_APCP_mean +GRID_STAT_OUTPUT_PREFIX = {ENV[MODEL]}_APCP_03h_{OBTYPE}_mean + +# Set grid to verify on +GRID_STAT_REGRID_TO_GRID = FCST + +# list of variables to compare +# thresholds in mm, equal to .01",.02",.05",.10",.25",.50",.75",1.0" +FCST_VAR1_NAME = APCP_A3_ENS_MEAN +FCST_VAR1_LEVELS = A3 + +BOTH_VAR1_THRESH = gt0.0,ge0.254,ge0.508,ge1.27,ge2.54,ge3.810,ge6.350 + +OBS_VAR1_NAME = APCP +OBS_VAR1_LEVELS = A3 + +# Neighborhood shape and widths +GRID_STAT_NEIGHBORHOOD_SHAPE = SQUARE +GRID_STAT_NEIGHBORHOOD_WIDTH = 3,5,7 + +# Forecast data description variables +FCST_IS_PROB = False + diff --git a/ush/templates/parm/metplus/GridStat_APCP03h_prob.conf b/ush/templates/parm/metplus/GridStat_APCP03h_prob.conf new file mode 100644 index 000000000..984eba7f0 --- /dev/null +++ b/ush/templates/parm/metplus/GridStat_APCP03h_prob.conf @@ -0,0 +1,138 @@ +# Ensemble Prob Grid to Grid Precipitation Example + +[dir] +# Input and output data directories +INPUT_BASE = {ENV[INPUT_BASE]} +OUTPUT_BASE = {ENV[EXPTDIR]} + +FCST_GRID_STAT_INPUT_DIR = {INPUT_BASE} +OBS_GRID_STAT_INPUT_DIR = {ENV[EXPTDIR]}/metprd/pcp_combine + +GRID_STAT_OUTPUT_DIR = {OUTPUT_BASE} + +LOG_DIR = {OUTPUT_BASE}/log + +STAGING_DIR = {OUTPUT_BASE}/stage/APCP_03h_prob + +[filename_templates] +# format of filenames +FCST_GRID_STAT_INPUT_TEMPLATE = ensemble_stat_{ENV[MODEL]}_APCP_{ENV[acc]}_{OBTYPE}_{valid?fmt=%Y%m%d}_{valid?fmt=%H%M%S}V_ens.nc + +OBS_GRID_STAT_INPUT_TEMPLATE = {OBS_GRID_STAT_INPUT_DIR}/{valid?fmt=%Y%m%d}/ccpa.t{valid?fmt=%H}z.hrap.conus.gb2_a03h + +GRID_STAT_OUTPUT_TEMPLATE = {init?fmt=%Y%m%d%H}/metprd/ensemble_stat_prob + +# Location of MET poly files +GRID_STAT_VERIFICATION_MASK_TEMPLATE = {MET_INSTALL_DIR}/share/met/poly/CONUS.poly + +# Specify the name of the metplus.log file +LOG_METPLUS = {LOG_DIR}/metplus.log.{ENV[LOG_SUFFIX]} + +# Specify where the location and name of the final metplus_final.conf +METPLUS_CONF={OUTPUT_BASE}/{ENV[CDATE]}/metprd/ensemble_stat_prob/metplus_final.APCP03.conf + +[config] +# List of applications to run +PROCESS_LIST = GridStat + +# time looping - options are INIT, VALID, RETRO, and REALTIME +LOOP_BY = INIT + +# Start time for METplus run +INIT_BEG = {ENV[CDATE]} +# End time for METplus run +INIT_END = {ENV[CDATE]} +# Increment between METplus runs in seconds. Must be > 60 +INIT_INCREMENT = 3600 +INIT_TIME_FMT = %Y%m%d%H + +# list of forecast leads to process. +# LEAD_SEQ = begin_end_incr(1,24,1) +LEAD_SEQ = {ENV[fhr_list]} + +# Options are times, processes +# times = run all items in the PROCESS_LIST for a single initialization +# time, then repeat until all times have been evaluated. +# processes = run each item in the PROCESS_LIST for all times +# specified, then repeat for the next item in the PROCESS_LIST. +LOOP_ORDER = times + +# Model to verify +MODEL = {ENV[MODEL]}_prob + +# Set obtype to vx +OBTYPE = CCPA + +# location of grid_stat MET config file +GRID_STAT_CONFIG_FILE = {CONFIG_DIR}/GridStatConfig_APCP_prob +GRID_STAT_OUTPUT_PREFIX = {ENV[MODEL]}_APCP_03h_{OBTYPE}_prob + +# Set grid to verify on +GRID_STAT_REGRID_TO_GRID = FCST + +# list of variables to compare +# thresholds in mm, equal to .01",.02",.05",.10",.25",.50",.75",1.0" +FCST_VAR1_NAME = APCP_A3_ENS_FREQ_gt0.0 +FCST_VAR1_LEVELS = A03 +FCST_VAR1_THRESH = ==0.1 + +OBS_VAR1_NAME = APCP +OBS_VAR1_LEVELS = A03 +OBS_VAR1_THRESH = >0.0 + +FCST_VAR2_NAME = APCP_A3_ENS_FREQ_ge0.254 +FCST_VAR2_LEVELS = A03 +FCST_VAR2_THRESH = ==0.1 + +OBS_VAR2_NAME = APCP +OBS_VAR2_LEVELS = A03 +OBS_VAR2_THRESH = >=0.254 + +FCST_VAR3_NAME = APCP_A3_ENS_FREQ_ge0.508 +FCST_VAR3_LEVELS = A03 +FCST_VAR3_THRESH = ==0.1 + +OBS_VAR3_NAME = APCP +OBS_VAR3_LEVELS = A03 +OBS_VAR3_THRESH = >=0.508 + +FCST_VAR4_NAME = APCP_A3_ENS_FREQ_ge1.27 +FCST_VAR4_LEVELS = A03 +FCST_VAR4_THRESH = ==0.1 + +OBS_VAR4_NAME = APCP +OBS_VAR4_LEVELS = A03 +OBS_VAR4_THRESH = >=1.27 + +FCST_VAR5_NAME = APCP_A3_ENS_FREQ_ge2.54 +FCST_VAR5_LEVELS = A03 +FCST_VAR5_THRESH = ==0.1 + +OBS_VAR5_NAME = APCP +OBS_VAR5_LEVELS = A03 +OBS_VAR5_THRESH = >=2.54 + +FCST_VAR6_NAME = APCP_A3_ENS_FREQ_ge3.810 +FCST_VAR6_LEVELS = A03 +FCST_VAR6_THRESH = ==0.1 + +OBS_VAR6_NAME = APCP +OBS_VAR6_LEVELS = A03 +OBS_VAR6_THRESH = >=3.810 + +FCST_VAR7_NAME = APCP_A3_ENS_FREQ_ge6.350 +FCST_VAR7_LEVELS = A03 +FCST_VAR7_THRESH = ==0.1 + +OBS_VAR7_NAME = APCP +OBS_VAR7_LEVELS = A03 +OBS_VAR7_THRESH = >=6.350 + + +# Neighborhood shape and widths +GRID_STAT_NEIGHBORHOOD_SHAPE = SQUARE +GRID_STAT_NEIGHBORHOOD_WIDTH = 3,5,7 + +# Forecast data description variables +FCST_IS_PROB = True +FCST_PROB_IN_GRIB_PDS = False diff --git a/ush/templates/parm/metplus/GridStat_APCP06h_mean.conf b/ush/templates/parm/metplus/GridStat_APCP06h_mean.conf new file mode 100644 index 000000000..b10707f0f --- /dev/null +++ b/ush/templates/parm/metplus/GridStat_APCP06h_mean.conf @@ -0,0 +1,89 @@ +# Ensemble Mean Grid to Grid Precipitation Example + +[dir] +# Input and output data directories +INPUT_BASE = {ENV[INPUT_BASE]} +OUTPUT_BASE = {ENV[EXPTDIR]} + +FCST_GRID_STAT_INPUT_DIR = {INPUT_BASE} +OBS_GRID_STAT_INPUT_DIR = {ENV[EXPTDIR]}/metprd/pcp_combine + +GRID_STAT_OUTPUT_DIR = {OUTPUT_BASE} + +LOG_DIR = {OUTPUT_BASE}/log + +STAGING_DIR = {OUTPUT_BASE}/stage/APCP_06h_mean + +[filename_templates] +# format of filenames +FCST_GRID_STAT_INPUT_TEMPLATE = ensemble_stat_{ENV[MODEL]}_APCP_{ENV[acc]}_{OBTYPE}_{valid?fmt=%Y%m%d}_{valid?fmt=%H%M%S}V_ens.nc + +OBS_GRID_STAT_INPUT_TEMPLATE = {OBS_GRID_STAT_INPUT_DIR}/{valid?fmt=%Y%m%d}/ccpa.t{valid?fmt=%H}z.hrap.conus.gb2_a06h + +GRID_STAT_OUTPUT_TEMPLATE = {init?fmt=%Y%m%d%H}/metprd/ensemble_stat_mean + +# Location of MET poly files +GRID_STAT_VERIFICATION_MASK_TEMPLATE = {MET_INSTALL_DIR}/share/met/poly/CONUS.poly + +# Specify the name of the metplus.log file +LOG_METPLUS = {LOG_DIR}/metplus.log.{ENV[LOG_SUFFIX]} + +# Specify where the location and name of the final metplus_final.conf +METPLUS_CONF={OUTPUT_BASE}/{ENV[CDATE]}/metprd/ensemble_stat_mean/metplus_final.APCP06.conf + +[config] +# List of applications to run +PROCESS_LIST = GridStat + +# time looping - options are INIT, VALID, RETRO, and REALTIME +LOOP_BY = INIT + +# Start time for METplus run +INIT_BEG = {ENV[CDATE]} +# End time for METplus run +INIT_END = {ENV[CDATE]} +# Increment between METplus runs in seconds. Must be > 60 +INIT_INCREMENT = 3600 +INIT_TIME_FMT = %Y%m%d%H + +# list of forecast leads to process. +# LEAD_SEQ = begin_end_incr(1,24,1) +LEAD_SEQ = {ENV[fhr_list]} + +# Options are times, processes +# times = run all items in the PROCESS_LIST for a single initialization +# time, then repeat until all times have been evaluated. +# processes = run each item in the PROCESS_LIST for all times +# specified, then repeat for the next item in the PROCESS_LIST. +LOOP_ORDER = times + +# Model to verify +MODEL = {ENV[MODEL]}_mean + +# Set obtype to vx +OBTYPE = CCPA + +# location of grid_stat MET config file +GRID_STAT_CONFIG_FILE = {CONFIG_DIR}/GridStatConfig_APCP_mean +GRID_STAT_OUTPUT_PREFIX = {ENV[MODEL]}_APCP_06h_{OBTYPE}_mean + +# Set grid to verify on +GRID_STAT_REGRID_TO_GRID = FCST + +# list of variables to compare +# thresholds in mm, equal to .01",.02",.05",.10",.25",.50",.75",1.0" +FCST_VAR1_NAME = APCP_A6_ENS_MEAN +FCST_VAR1_LEVELS = A06 + +BOTH_VAR1_THRESH = gt0.0,ge0.254,ge0.508,ge1.27,ge2.54,ge3.810,ge6.350 + +OBS_VAR1_NAME = APCP +OBS_VAR1_LEVELS = A06 + +# Neighborhood shape and widths +GRID_STAT_NEIGHBORHOOD_SHAPE = SQUARE +GRID_STAT_NEIGHBORHOOD_WIDTH = 3,5,7 + +# Forecast data description variables +FCST_IS_PROB = False + diff --git a/ush/templates/parm/metplus/GridStat_APCP06h_prob.conf b/ush/templates/parm/metplus/GridStat_APCP06h_prob.conf new file mode 100644 index 000000000..7c5e08672 --- /dev/null +++ b/ush/templates/parm/metplus/GridStat_APCP06h_prob.conf @@ -0,0 +1,138 @@ +# Ensemble Prob Grid to Grid Precipitation Example + +[dir] +# Input and output data directories +INPUT_BASE = {ENV[INPUT_BASE]} +OUTPUT_BASE = {ENV[EXPTDIR]} + +FCST_GRID_STAT_INPUT_DIR = {INPUT_BASE} +OBS_GRID_STAT_INPUT_DIR = {ENV[EXPTDIR]}/metprd/pcp_combine + +GRID_STAT_OUTPUT_DIR = {OUTPUT_BASE} + +LOG_DIR = {OUTPUT_BASE}/log + +STAGING_DIR = {OUTPUT_BASE}/stage/APCP_06h_prob + +[filename_templates] +# format of filenames +FCST_GRID_STAT_INPUT_TEMPLATE = ensemble_stat_{ENV[MODEL]}_APCP_{ENV[acc]}_{OBTYPE}_{valid?fmt=%Y%m%d}_{valid?fmt=%H%M%S}V_ens.nc + +OBS_GRID_STAT_INPUT_TEMPLATE = {OBS_GRID_STAT_INPUT_DIR}/{valid?fmt=%Y%m%d}/ccpa.t{valid?fmt=%H}z.hrap.conus.gb2_a06h + +GRID_STAT_OUTPUT_TEMPLATE = {init?fmt=%Y%m%d%H}/metprd/ensemble_stat_prob + +# Location of MET poly files +GRID_STAT_VERIFICATION_MASK_TEMPLATE = {MET_INSTALL_DIR}/share/met/poly/CONUS.poly + +# Specify the name of the metplus.log file +LOG_METPLUS = {LOG_DIR}/metplus.log.{ENV[LOG_SUFFIX]} + +# Specify where the location and name of the final metplus_final.conf +METPLUS_CONF={OUTPUT_BASE}/{ENV[CDATE]}/metprd/ensemble_stat_prob/metplus_final.APCP06.conf + +[config] +# List of applications to run +PROCESS_LIST = GridStat + +# time looping - options are INIT, VALID, RETRO, and REALTIME +LOOP_BY = INIT + +# Start time for METplus run +INIT_BEG = {ENV[CDATE]} +# End time for METplus run +INIT_END = {ENV[CDATE]} +# Increment between METplus runs in seconds. Must be > 60 +INIT_INCREMENT = 3600 +INIT_TIME_FMT = %Y%m%d%H + +# list of forecast leads to process. +# LEAD_SEQ = begin_end_incr(1,24,1) +LEAD_SEQ = {ENV[fhr_list]} + +# Options are times, processes +# times = run all items in the PROCESS_LIST for a single initialization +# time, then repeat until all times have been evaluated. +# processes = run each item in the PROCESS_LIST for all times +# specified, then repeat for the next item in the PROCESS_LIST. +LOOP_ORDER = times + +# Model to verify +MODEL = {ENV[MODEL]}_prob + +# Set obtype to vx +OBTYPE = CCPA + +# location of grid_stat MET config file +GRID_STAT_CONFIG_FILE = {CONFIG_DIR}/GridStatConfig_APCP_prob +GRID_STAT_OUTPUT_PREFIX = {ENV[MODEL]}_APCP_06h_{OBTYPE}_prob + +# Set grid to verify on +GRID_STAT_REGRID_TO_GRID = FCST + +# list of variables to compare +# thresholds in mm, equal to .01",.02",.05",.10",.25",.50",.75",1.0" +FCST_VAR1_NAME = APCP_A6_ENS_FREQ_gt0.0 +FCST_VAR1_LEVELS = A06 +FCST_VAR1_THRESH = ==0.1 + +OBS_VAR1_NAME = APCP +OBS_VAR1_LEVELS = A06 +OBS_VAR1_THRESH = >0.0 + +FCST_VAR2_NAME = APCP_A6_ENS_FREQ_ge0.254 +FCST_VAR2_LEVELS = A06 +FCST_VAR2_THRESH = ==0.1 + +OBS_VAR2_NAME = APCP +OBS_VAR2_LEVELS = A06 +OBS_VAR2_THRESH = >=0.254 + +FCST_VAR3_NAME = APCP_A6_ENS_FREQ_ge0.508 +FCST_VAR3_LEVELS = A06 +FCST_VAR3_THRESH = ==0.1 + +OBS_VAR3_NAME = APCP +OBS_VAR3_LEVELS = A06 +OBS_VAR3_THRESH = >=0.508 + +FCST_VAR4_NAME = APCP_A6_ENS_FREQ_ge1.27 +FCST_VAR4_LEVELS = A06 +FCST_VAR4_THRESH = ==0.1 + +OBS_VAR4_NAME = APCP +OBS_VAR4_LEVELS = A06 +OBS_VAR4_THRESH = >=1.27 + +FCST_VAR5_NAME = APCP_A6_ENS_FREQ_ge2.54 +FCST_VAR5_LEVELS = A06 +FCST_VAR5_THRESH = ==0.1 + +OBS_VAR5_NAME = APCP +OBS_VAR5_LEVELS = A06 +OBS_VAR5_THRESH = >=2.54 + +FCST_VAR6_NAME = APCP_A6_ENS_FREQ_ge3.810 +FCST_VAR6_LEVELS = A06 +FCST_VAR6_THRESH = ==0.1 + +OBS_VAR6_NAME = APCP +OBS_VAR6_LEVELS = A06 +OBS_VAR6_THRESH = >=3.810 + +FCST_VAR7_NAME = APCP_A6_ENS_FREQ_ge6.350 +FCST_VAR7_LEVELS = A06 +FCST_VAR7_THRESH = ==0.1 + +OBS_VAR7_NAME = APCP +OBS_VAR7_LEVELS = A06 +OBS_VAR7_THRESH = >=6.350 + + +# Neighborhood shape and widths +GRID_STAT_NEIGHBORHOOD_SHAPE = SQUARE +GRID_STAT_NEIGHBORHOOD_WIDTH = 3,5,7 + +# Forecast data description variables +FCST_IS_PROB = True +FCST_PROB_IN_GRIB_PDS = False diff --git a/ush/templates/parm/metplus/GridStat_APCP24h_mean.conf b/ush/templates/parm/metplus/GridStat_APCP24h_mean.conf new file mode 100644 index 000000000..aa0c53e21 --- /dev/null +++ b/ush/templates/parm/metplus/GridStat_APCP24h_mean.conf @@ -0,0 +1,89 @@ +# Ensemble Mean Grid to Grid Precipitation Example + +[dir] +# Input and output data directories +INPUT_BASE = {ENV[INPUT_BASE]} +OUTPUT_BASE = {ENV[EXPTDIR]} + +FCST_GRID_STAT_INPUT_DIR = {INPUT_BASE} +OBS_GRID_STAT_INPUT_DIR = {ENV[EXPTDIR]}/metprd/pcp_combine + +GRID_STAT_OUTPUT_DIR = {OUTPUT_BASE} + +LOG_DIR = {OUTPUT_BASE}/log + +STAGING_DIR = {OUTPUT_BASE}/stage/APCP_24h_mean + +[filename_templates] +# format of filenames +FCST_GRID_STAT_INPUT_TEMPLATE = ensemble_stat_{ENV[MODEL]}_APCP_{ENV[acc]}_{OBTYPE}_{valid?fmt=%Y%m%d}_{valid?fmt=%H%M%S}V_ens.nc + +OBS_GRID_STAT_INPUT_TEMPLATE = {OBS_GRID_STAT_INPUT_DIR}/{valid?fmt=%Y%m%d}/ccpa.t{valid?fmt=%H}z.hrap.conus.gb2_a24h + +GRID_STAT_OUTPUT_TEMPLATE = {init?fmt=%Y%m%d%H}/metprd/ensemble_stat_mean + +# Location of MET poly files +GRID_STAT_VERIFICATION_MASK_TEMPLATE = {MET_INSTALL_DIR}/share/met/poly/CONUS.poly + +# Specify the name of the metplus.log file +LOG_METPLUS = {LOG_DIR}/metplus.log.{ENV[LOG_SUFFIX]} + +# Specify where the location and name of the final metplus_final.conf +METPLUS_CONF={OUTPUT_BASE}/{ENV[CDATE]}/metprd/ensemble_stat_mean/metplus_final.APCP24.conf + +[config] +# List of applications to run +PROCESS_LIST = GridStat + +# time looping - options are INIT, VALID, RETRO, and REALTIME +LOOP_BY = INIT + +# Start time for METplus run +INIT_BEG = {ENV[CDATE]} +# End time for METplus run +INIT_END = {ENV[CDATE]} +# Increment between METplus runs in seconds. Must be > 60 +INIT_INCREMENT = 3600 +INIT_TIME_FMT = %Y%m%d%H + +# list of forecast leads to process. +# LEAD_SEQ = begin_end_incr(1,24,1) +LEAD_SEQ = {ENV[fhr_list]} + +# Options are times, processes +# times = run all items in the PROCESS_LIST for a single initialization +# time, then repeat until all times have been evaluated. +# processes = run each item in the PROCESS_LIST for all times +# specified, then repeat for the next item in the PROCESS_LIST. +LOOP_ORDER = times + +# Model to verify +MODEL = {ENV[MODEL]}_mean + +# Set obtype to vx +OBTYPE = CCPA + +# location of grid_stat MET config file +GRID_STAT_CONFIG_FILE = {CONFIG_DIR}/GridStatConfig_APCP_mean +GRID_STAT_OUTPUT_PREFIX = {ENV[MODEL]}_APCP_24h_{OBTYPE}_mean + +# Set grid to verify on +GRID_STAT_REGRID_TO_GRID = FCST + +# list of variables to compare +# thresholds in mm, equal to .01",.02",.05",.10",.25",.50",.75",1.0" +FCST_VAR1_NAME = APCP_A24_ENS_MEAN +FCST_VAR1_LEVELS = A24 + +BOTH_VAR1_THRESH = gt0.0,ge0.254,ge0.508,ge1.27,ge2.54,ge3.810,ge6.350 + +OBS_VAR1_NAME = APCP +OBS_VAR1_LEVELS = A24 + +# Neighborhood shape and widths +GRID_STAT_NEIGHBORHOOD_SHAPE = SQUARE +GRID_STAT_NEIGHBORHOOD_WIDTH = 3,5,7 + +# Forecast data description variables +FCST_IS_PROB = False + diff --git a/ush/templates/parm/metplus/GridStat_APCP24h_prob.conf b/ush/templates/parm/metplus/GridStat_APCP24h_prob.conf new file mode 100644 index 000000000..26d6e6368 --- /dev/null +++ b/ush/templates/parm/metplus/GridStat_APCP24h_prob.conf @@ -0,0 +1,138 @@ +# Ensemble Prob Grid to Grid Precipitation Example + +[dir] +# Input and output data directories +INPUT_BASE = {ENV[INPUT_BASE]} +OUTPUT_BASE = {ENV[EXPTDIR]} + +FCST_GRID_STAT_INPUT_DIR = {INPUT_BASE} +OBS_GRID_STAT_INPUT_DIR = {ENV[EXPTDIR]}/metprd/pcp_combine + +GRID_STAT_OUTPUT_DIR = {OUTPUT_BASE} + +LOG_DIR = {OUTPUT_BASE}/log + +STAGING_DIR = {OUTPUT_BASE}/stage/APCP_24h_prob + +[filename_templates] +# format of filenames +FCST_GRID_STAT_INPUT_TEMPLATE = ensemble_stat_{ENV[MODEL]}_APCP_{ENV[acc]}_{OBTYPE}_{valid?fmt=%Y%m%d}_{valid?fmt=%H%M%S}V_ens.nc + +OBS_GRID_STAT_INPUT_TEMPLATE = {OBS_GRID_STAT_INPUT_DIR}/{valid?fmt=%Y%m%d}/ccpa.t{valid?fmt=%H}z.hrap.conus.gb2_a24h + +GRID_STAT_OUTPUT_TEMPLATE = {init?fmt=%Y%m%d%H}/metprd/ensemble_stat_prob + +# Location of MET poly files +GRID_STAT_VERIFICATION_MASK_TEMPLATE = {MET_INSTALL_DIR}/share/met/poly/CONUS.poly + +# Specify the name of the metplus.log file +LOG_METPLUS = {LOG_DIR}/metplus.log.{ENV[LOG_SUFFIX]} + +# Specify where the location and name of the final metplus_final.conf +METPLUS_CONF={OUTPUT_BASE}/{ENV[CDATE]}/metprd/ensemble_stat_prob/metplus_final.APCP24.conf + +[config] +# List of applications to run +PROCESS_LIST = GridStat + +# time looping - options are INIT, VALID, RETRO, and REALTIME +LOOP_BY = INIT + +# Start time for METplus run +INIT_BEG = {ENV[CDATE]} +# End time for METplus run +INIT_END = {ENV[CDATE]} +# Increment between METplus runs in seconds. Must be > 60 +INIT_INCREMENT = 3600 +INIT_TIME_FMT = %Y%m%d%H + +# list of forecast leads to process. +# LEAD_SEQ = begin_end_incr(1,24,1) +LEAD_SEQ = {ENV[fhr_list]} + +# Options are times, processes +# times = run all items in the PROCESS_LIST for a single initialization +# time, then repeat until all times have been evaluated. +# processes = run each item in the PROCESS_LIST for all times +# specified, then repeat for the next item in the PROCESS_LIST. +LOOP_ORDER = times + +# Model to verify +MODEL = {ENV[MODEL]}_prob + +# Set obtype to vx +OBTYPE = CCPA + +# location of grid_stat MET config file +GRID_STAT_CONFIG_FILE = {CONFIG_DIR}/GridStatConfig_APCP_prob +GRID_STAT_OUTPUT_PREFIX = {ENV[MODEL]}_APCP_24h_{OBTYPE}_prob + +# Set grid to verify on +GRID_STAT_REGRID_TO_GRID = FCST + +# list of variables to compare +# thresholds in mm, equal to .01",.02",.05",.10",.25",.50",.75",1.0" +FCST_VAR1_NAME = APCP_A24_ENS_FREQ_gt0.0 +FCST_VAR1_LEVELS = A24 +FCST_VAR1_THRESH = ==0.1 + +OBS_VAR1_NAME = APCP +OBS_VAR1_LEVELS = A24 +OBS_VAR1_THRESH = >0.0 + +FCST_VAR2_NAME = APCP_A24_ENS_FREQ_ge0.254 +FCST_VAR2_LEVELS = A24 +FCST_VAR2_THRESH = ==0.1 + +OBS_VAR2_NAME = APCP +OBS_VAR2_LEVELS = A24 +OBS_VAR2_THRESH = >=0.254 + +FCST_VAR3_NAME = APCP_A24_ENS_FREQ_ge0.508 +FCST_VAR3_LEVELS = A24 +FCST_VAR3_THRESH = ==0.1 + +OBS_VAR3_NAME = APCP +OBS_VAR3_LEVELS = A24 +OBS_VAR3_THRESH = >=0.508 + +FCST_VAR4_NAME = APCP_A24_ENS_FREQ_ge1.27 +FCST_VAR4_LEVELS = A24 +FCST_VAR4_THRESH = ==0.1 + +OBS_VAR4_NAME = APCP +OBS_VAR4_LEVELS = A24 +OBS_VAR4_THRESH = >=1.27 + +FCST_VAR5_NAME = APCP_A24_ENS_FREQ_ge2.54 +FCST_VAR5_LEVELS = A24 +FCST_VAR5_THRESH = ==0.1 + +OBS_VAR5_NAME = APCP +OBS_VAR5_LEVELS = A24 +OBS_VAR5_THRESH = >=2.54 + +FCST_VAR6_NAME = APCP_A24_ENS_FREQ_ge3.810 +FCST_VAR6_LEVELS = A24 +FCST_VAR6_THRESH = ==0.1 + +OBS_VAR6_NAME = APCP +OBS_VAR6_LEVELS = A24 +OBS_VAR6_THRESH = >=3.810 + +FCST_VAR7_NAME = APCP_A24_ENS_FREQ_ge6.350 +FCST_VAR7_LEVELS = A24 +FCST_VAR7_THRESH = ==0.1 + +OBS_VAR7_NAME = APCP +OBS_VAR7_LEVELS = A24 +OBS_VAR7_THRESH = >=6.350 + + +# Neighborhood shape and widths +GRID_STAT_NEIGHBORHOOD_SHAPE = SQUARE +GRID_STAT_NEIGHBORHOOD_WIDTH = 3,5,7 + +# Forecast data description variables +FCST_IS_PROB = True +FCST_PROB_IN_GRIB_PDS = False diff --git a/ush/templates/parm/metplus/GridStat_REFC_mean.conf b/ush/templates/parm/metplus/GridStat_REFC_mean.conf new file mode 100644 index 000000000..098e2c010 --- /dev/null +++ b/ush/templates/parm/metplus/GridStat_REFC_mean.conf @@ -0,0 +1,94 @@ +# Ensemble Mean Composite Reflectivity Example + +[dir] +# Input and output data directories +INPUT_BASE = {ENV[INPUT_BASE]} +OUTPUT_BASE = {ENV[EXPTDIR]} + +FCST_GRID_STAT_INPUT_DIR = {INPUT_BASE} +OBS_GRID_STAT_INPUT_DIR = {ENV[OBS_DIR]} + +GRID_STAT_OUTPUT_DIR = {OUTPUT_BASE} + +LOG_DIR = {OUTPUT_BASE}/log + +STAGING_DIR = {OUTPUT_BASE}/stage/REFC_mean + +[filename_templates] +# format of filenames +FCST_GRID_STAT_INPUT_TEMPLATE = ensemble_stat_{ENV[MODEL]}_REFC_{OBTYPE}_{valid?fmt=%Y%m%d}_{valid?fmt=%H%M%S}V_ens.nc + +# ANLYS +OBS_GRID_STAT_INPUT_TEMPLATE = {valid?fmt=%Y%m%d}/MergedReflectivityQCComposite_00.50_{valid?fmt=%Y%m%d}-{valid?fmt=%H}0000.grib2 + +GRID_STAT_OUTPUT_TEMPLATE = {init?fmt=%Y%m%d%H}/metprd/ensemble_stat_mean + +# Location of MET poly files +GRID_STAT_VERIFICATION_MASK_TEMPLATE = {MET_INSTALL_DIR}/share/met/poly/CONUS.poly + +# Specify the name of the metplus.log file +LOG_METPLUS = {LOG_DIR}/metplus.log.{ENV[LOG_SUFFIX]} + +# Specify where the location and name of the final metplus_final.conf +METPLUS_CONF={OUTPUT_BASE}/{ENV[CDATE]}/metprd/ensemble_stat_mean/metplus_final.REFC.conf + +[config] +# List of applications to run +PROCESS_LIST = GridStat + +# time looping - options are INIT, VALID, RETRO, and REALTIME +LOOP_BY = INIT + +# Start time for METplus run +INIT_BEG = {ENV[CDATE]} +# End time for METplus run +INIT_END = {ENV[CDATE]} +# Increment between METplus runs in seconds. Must be > 60 +INIT_INCREMENT = 3600 +INIT_TIME_FMT = %Y%m%d%H + +# list of forecast leads to process. +#LEAD_SEQ = begin_end_incr(1,24,1) +LEAD_SEQ = {ENV[fhr_list]} + +# Options are times, processes +# times = run all items in the PROCESS_LIST for a single initialization +# time, then repeat until all times have been evaluated. +# processes = run each item in the PROCESS_LIST for all times +# specified, then repeat for the next item in the PROCESS_LIST. +LOOP_ORDER = times + +# Model to verify +MODEL = {ENV[MODEL]}_mean + +# Set obtype to vx +OBTYPE = MRMS +OBS_NATIVE_DATA_TYPE = GRIB + +# location of grid_stat MET config file +GRID_STAT_CONFIG_FILE = {CONFIG_DIR}/GridStatConfig_REFC_mean +GRID_STAT_OUTPUT_PREFIX = {ENV[MODEL]}_REFC_{OBTYPE}_mean + +# Set grid to verify on +GRID_STAT_REGRID_TO_GRID = FCST + +# Forecast/Observation variable Information +FCST_VAR1_NAME = REFC_L0_ENS_MEAN +FCST_VAR1_LEVELS = L0 + +BOTH_VAR1_THRESH = ge20, ge30, ge40, ge50 + +OBS_VAR1_NAME = MergedReflectivityQCComposite +OBS_VAR1_LEVELS = Z500 +OBS_VAR1_OPTIONS = censor_thresh = lt-20; censor_val = -20.0; cnt_thresh = [ >15 ]; cnt_logic = UNION; + +OBS_GRID_STAT_FILE_WINDOW_BEGIN = -300 +OBS_GRID_STAT_FILE_WINDOW_END = 300 + +# Neighborhood shape and width +GRID_STAT_NEIGHBORHOOD_SHAPE = SQUARE +GRID_STAT_NEIGHBORHOOD_WIDTH = 3, 5, 7 + +# Forecast data description variables +FCST_IS_PROB = False + diff --git a/ush/templates/parm/metplus/GridStat_REFC_prob.conf b/ush/templates/parm/metplus/GridStat_REFC_prob.conf new file mode 100644 index 000000000..d6ad7452a --- /dev/null +++ b/ush/templates/parm/metplus/GridStat_REFC_prob.conf @@ -0,0 +1,122 @@ +# Ensemble Prob Grid to Grid Composite Reflectivity Example + +[dir] +# Input and output data directories +INPUT_BASE = {ENV[INPUT_BASE]} +OUTPUT_BASE = {ENV[EXPTDIR]} + +FCST_GRID_STAT_INPUT_DIR = {INPUT_BASE} +OBS_GRID_STAT_INPUT_DIR = {ENV[OBS_DIR]} + +GRID_STAT_OUTPUT_DIR = {OUTPUT_BASE} + +LOG_DIR = {OUTPUT_BASE}/log + +STAGING_DIR = {OUTPUT_BASE}/stage/REFC_prob + +[filename_templates] +# format of filenames +FCST_GRID_STAT_INPUT_TEMPLATE = ensemble_stat_{ENV[MODEL]}_REFC_{OBTYPE}_{valid?fmt=%Y%m%d}_{valid?fmt=%H%M%S}V_ens.nc + +OBS_GRID_STAT_INPUT_TEMPLATE = {valid?fmt=%Y%m%d}/MergedReflectivityQCComposite_00.50_{valid?fmt=%Y%m%d}-{valid?fmt=%H}0000.grib2 + +GRID_STAT_OUTPUT_TEMPLATE = {init?fmt=%Y%m%d%H}/metprd/ensemble_stat_prob + +# Location of MET poly files +GRID_STAT_VERIFICATION_MASK_TEMPLATE = {MET_INSTALL_DIR}/share/met/poly/CONUS.poly + +# Specify the name of the metplus.log file +LOG_METPLUS = {LOG_DIR}/metplus.log.{ENV[LOG_SUFFIX]} + +# Specify where the location and name of the final metplus_final.conf +METPLUS_CONF={OUTPUT_BASE}/{ENV[CDATE]}/metprd/ensemble_stat_prob/metplus_final.REFC.conf + +[config] +# List of applications to run +PROCESS_LIST = GridStat + +# time looping - options are INIT, VALID, RETRO, and REALTIME +LOOP_BY = INIT + +# Start time for METplus run +INIT_BEG = {ENV[CDATE]} +# End time for METplus run +INIT_END = {ENV[CDATE]} +# Increment between METplus runs in seconds. Must be > 60 +INIT_INCREMENT = 3600 +INIT_TIME_FMT = %Y%m%d%H + +# list of forecast leads to process. +# LEAD_SEQ = begin_end_incr(1,24,1) +LEAD_SEQ = {ENV[fhr_list]} + +# Options are times, processes +# times = run all items in the PROCESS_LIST for a single initialization +# time, then repeat until all times have been evaluated. +# processes = run each item in the PROCESS_LIST for all times +# specified, then repeat for the next item in the PROCESS_LIST. +LOOP_ORDER = times + +# Model to verify +MODEL = {ENV[MODEL]}_prob +FCST_NATIVE_DATA_TYPE = GRIB + +# Set obtype to vx +OBTYPE = MRMS +OBS_NATIVE_DATA_TYPE = GRIB + +# location of grid_stat MET config file +GRID_STAT_CONFIG_FILE = {CONFIG_DIR}/GridStatConfig_REFC_prob +GRID_STAT_OUTPUT_PREFIX = {ENV[MODEL]}_REFC_{OBTYPE}_prob + +# run pcp_combine on forecast/obs data? +FCST_PCP_COMBINE_RUN = False +OBS_PCP_COMBINE_RUN = False + +# Set grid to verify on +GRID_STAT_REGRID_TO_GRID = FCST + +# list of variables to compare +FCST_VAR1_NAME = REFC_L0_ENS_FREQ_ge20 +FCST_VAR1_LEVELS = L0 +FCST_VAR1_THRESH = ==0.1 + +OBS_VAR1_NAME = MergedReflectivityQCComposite +OBS_VAR1_LEVELS = Z500 +OBS_VAR1_THRESH = >=20.0 +OBS_VAR1_OPTIONS = censor_thresh = lt-20; censor_val = -20.0; cnt_thresh = [ >15 ]; cnt_logic = UNION; + +FCST_VAR2_NAME = REFC_L0_ENS_FREQ_ge30 +FCST_VAR2_LEVELS = L0 +FCST_VAR2_THRESH = ==0.1 + +OBS_VAR2_NAME = MergedReflectivityQCComposite +OBS_VAR2_LEVELS = Z500 +OBS_VAR2_THRESH = >=30.0 +OBS_VAR2_OPTIONS = censor_thresh = lt-20; censor_val = -20.0; cnt_thresh = [ >15 ]; cnt_logic = UNION; + +FCST_VAR3_NAME = REFC_L0_ENS_FREQ_ge40 +FCST_VAR3_LEVELS = L0 +FCST_VAR3_THRESH = ==0.1 + +OBS_VAR3_NAME = MergedReflectivityQCComposite +OBS_VAR3_LEVELS = Z500 +OBS_VAR3_THRESH = >=40.0 +OBS_VAR3_OPTIONS = censor_thresh = lt-20; censor_val = -20.0; cnt_thresh = [ >15 ]; cnt_logic = UNION; + +FCST_VAR4_NAME = REFC_L0_ENS_FREQ_ge50 +FCST_VAR4_LEVELS = L0 +FCST_VAR4_THRESH = ==0.1 + +OBS_VAR4_NAME = MergedReflectivityQCComposite +OBS_VAR4_LEVELS = Z500 +OBS_VAR4_THRESH = >=50.0 +OBS_VAR4_OPTIONS = censor_thresh = lt-20; censor_val = -20.0; cnt_thresh = [ >15 ]; cnt_logic = UNION; + +# Neighborhood shape and widths +GRID_STAT_NEIGHBORHOOD_SHAPE = SQUARE +GRID_STAT_NEIGHBORHOOD_WIDTH = 3,5,7 + +# Forecast data description variables +FCST_IS_PROB = True +FCST_PROB_IN_GRIB_PDS = False diff --git a/ush/templates/parm/metplus/GridStat_RETOP_mean.conf b/ush/templates/parm/metplus/GridStat_RETOP_mean.conf new file mode 100644 index 000000000..6755ff2ee --- /dev/null +++ b/ush/templates/parm/metplus/GridStat_RETOP_mean.conf @@ -0,0 +1,95 @@ +# Ensemble Mean Composite RETOP Example + +[dir] +# Input and output data directories +INPUT_BASE = {ENV[INPUT_BASE]} +OUTPUT_BASE = {ENV[EXPTDIR]} + +FCST_GRID_STAT_INPUT_DIR = {INPUT_BASE} +OBS_GRID_STAT_INPUT_DIR = {ENV[OBS_DIR]} + +GRID_STAT_OUTPUT_DIR = {OUTPUT_BASE} + +LOG_DIR = {OUTPUT_BASE}/log + +STAGING_DIR = {OUTPUT_BASE}/stage/RETOP_mean + +[filename_templates] +# format of filenames +FCST_GRID_STAT_INPUT_TEMPLATE = ensemble_stat_{ENV[MODEL]}_RETOP_{OBTYPE}_{valid?fmt=%Y%m%d}_{valid?fmt=%H%M%S}V_ens.nc + +# ANLYS +OBS_GRID_STAT_INPUT_TEMPLATE = {valid?fmt=%Y%m%d}/EchoTop_18_00.50_{valid?fmt=%Y%m%d}-{valid?fmt=%H}0000.grib2 + +GRID_STAT_OUTPUT_TEMPLATE = {init?fmt=%Y%m%d%H}/metprd/ensemble_stat_mean + +# Location of MET poly files +GRID_STAT_VERIFICATION_MASK_TEMPLATE = {MET_INSTALL_DIR}/share/met/poly/CONUS.poly + +# Specify the name of the metplus.log file +LOG_METPLUS = {LOG_DIR}/metplus.log.{ENV[LOG_SUFFIX]} + +# Specify where the location and name of the final metplus_final.conf +METPLUS_CONF={OUTPUT_BASE}/{ENV[CDATE]}/metprd/ensemble_stat_mean/metplus_final.RETOP.conf + +[config] +# List of applications to run +PROCESS_LIST = GridStat + +# time looping - options are INIT, VALID, RETRO, and REALTIME +LOOP_BY = INIT + +# Start time for METplus run +INIT_BEG = {ENV[CDATE]} +# End time for METplus run +INIT_END = {ENV[CDATE]} +# Increment between METplus runs in seconds. Must be > 60 +INIT_INCREMENT = 3600 +INIT_TIME_FMT = %Y%m%d%H + +# list of forecast leads to process. +#LEAD_SEQ = begin_end_incr(1,24,1) +LEAD_SEQ = {ENV[fhr_list]} + +# Options are times, processes +# times = run all items in the PROCESS_LIST for a single initialization +# time, then repeat until all times have been evaluated. +# processes = run each item in the PROCESS_LIST for all times +# specified, then repeat for the next item in the PROCESS_LIST. +LOOP_ORDER = times + +# Model to verify +MODEL = {ENV[MODEL]}_mean + +# Set obtype to vx +OBTYPE = MRMS +OBS_NATIVE_DATA_TYPE = GRIB + +# location of grid_stat MET config file +GRID_STAT_CONFIG_FILE = {CONFIG_DIR}/GridStatConfig_REFC_mean +GRID_STAT_OUTPUT_PREFIX = {ENV[MODEL]}_RETOP_{OBTYPE}_mean + +# Set grid to verify on +GRID_STAT_REGRID_TO_GRID = FCST + +# Forecast/Observation variable Information +FCST_VAR1_NAME = RETOP_L0_ENS_MEAN +FCST_VAR1_LEVELS = L0 +FCST_VAR1_OPTIONS = convert(x) = M_to_KFT(x); + +BOTH_VAR1_THRESH = ge20, ge30, ge40, ge50 + +OBS_VAR1_NAME = EchoTop18 +OBS_VAR1_LEVELS = Z500 +OBS_VAR1_OPTIONS = censor_thresh = censor_thresh = lt-20.0; censor_val = -20.0; cnt_thresh = [ >15 ]; cnt_logic = UNION; convert(x) = KM_to_KFT(x); + +OBS_GRID_STAT_FILE_WINDOW_BEGIN = -300 +OBS_GRID_STAT_FILE_WINDOW_END = 300 + +# Neighborhood shape and width +GRID_STAT_NEIGHBORHOOD_SHAPE = SQUARE +GRID_STAT_NEIGHBORHOOD_WIDTH = 3, 5, 7 + +# Forecast data description variables +FCST_IS_PROB = False + diff --git a/ush/templates/parm/metplus/GridStat_RETOP_prob.conf b/ush/templates/parm/metplus/GridStat_RETOP_prob.conf new file mode 100644 index 000000000..41741a067 --- /dev/null +++ b/ush/templates/parm/metplus/GridStat_RETOP_prob.conf @@ -0,0 +1,122 @@ +# Ensemble Prob Grid to Grid Composite RETOP Example + +[dir] +# Input and output data directories +INPUT_BASE = {ENV[INPUT_BASE]} +OUTPUT_BASE = {ENV[EXPTDIR]} + +FCST_GRID_STAT_INPUT_DIR = {INPUT_BASE} +OBS_GRID_STAT_INPUT_DIR = {ENV[OBS_DIR]} + +GRID_STAT_OUTPUT_DIR = {OUTPUT_BASE} + +LOG_DIR = {OUTPUT_BASE}/log + +STAGING_DIR = {OUTPUT_BASE}/stage/RETOP_prob + +[filename_templates] +# format of filenames +FCST_GRID_STAT_INPUT_TEMPLATE = ensemble_stat_{ENV[MODEL]}_RETOP_{OBTYPE}_{valid?fmt=%Y%m%d}_{valid?fmt=%H%M%S}V_ens.nc + +OBS_GRID_STAT_INPUT_TEMPLATE = {valid?fmt=%Y%m%d}/EchoTop_18_00.50_{valid?fmt=%Y%m%d}-{valid?fmt=%H}0000.grib2 + +GRID_STAT_OUTPUT_TEMPLATE = {init?fmt=%Y%m%d%H}/metprd/ensemble_stat_prob + +# Location of MET poly files +GRID_STAT_VERIFICATION_MASK_TEMPLATE = {MET_INSTALL_DIR}/share/met/poly/CONUS.poly + +# Specify the name of the metplus.log file +LOG_METPLUS = {LOG_DIR}/metplus.log.{ENV[LOG_SUFFIX]} + +# Specify where the location and name of the final metplus_final.conf +METPLUS_CONF={OUTPUT_BASE}/{ENV[CDATE]}/metprd/ensemble_stat_prob/metplus_final.RETOP.conf + +[config] +# List of applications to run +PROCESS_LIST = GridStat + +# time looping - options are INIT, VALID, RETRO, and REALTIME +LOOP_BY = INIT + +# Start time for METplus run +INIT_BEG = {ENV[CDATE]} +# End time for METplus run +INIT_END = {ENV[CDATE]} +# Increment between METplus runs in seconds. Must be > 60 +INIT_INCREMENT = 3600 +INIT_TIME_FMT = %Y%m%d%H + +# list of forecast leads to process. +# LEAD_SEQ = begin_end_incr(1,24,1) +LEAD_SEQ = {ENV[fhr_list]} + +# Options are times, processes +# times = run all items in the PROCESS_LIST for a single initialization +# time, then repeat until all times have been evaluated. +# processes = run each item in the PROCESS_LIST for all times +# specified, then repeat for the next item in the PROCESS_LIST. +LOOP_ORDER = times + +# Model to verify +MODEL = {ENV[MODEL]}_prob +FCST_NATIVE_DATA_TYPE = GRIB + +# Set obtype to vx +OBTYPE = MRMS +OBS_NATIVE_DATA_TYPE = GRIB + +# location of grid_stat MET config file +GRID_STAT_CONFIG_FILE = {CONFIG_DIR}/GridStatConfig_REFC_prob +GRID_STAT_OUTPUT_PREFIX = {ENV[MODEL]}_RETOP_{OBTYPE}_prob + +# run pcp_combine on forecast/obs data? +FCST_PCP_COMBINE_RUN = False +OBS_PCP_COMBINE_RUN = False + +# Set grid to verify on +GRID_STAT_REGRID_TO_GRID = FCST + +# list of variables to compare +FCST_VAR1_NAME = RETOP_L0_ENS_FREQ_ge20 +FCST_VAR1_LEVELS = L0 +FCST_VAR1_THRESH = ==0.1 + +OBS_VAR1_NAME = EchoTop18 +OBS_VAR1_LEVELS = Z500 +OBS_VAR1_THRESH = >=20.0 +OBS_VAR1_OPTIONS = censor_thresh = lt-20.0; censor_val = -20.0; cnt_thresh = [ >15 ]; cnt_logic = UNION; convert(x) = KM_to_KFT(x); + +FCST_VAR2_NAME = RETOP_L0_ENS_FREQ_ge30 +FCST_VAR2_LEVELS = L0 +FCST_VAR2_THRESH = ==0.1 + +OBS_VAR2_NAME = EchoTop18 +OBS_VAR2_LEVELS = Z500 +OBS_VAR2_THRESH = >=30.0 +OBS_VAR2_OPTIONS = censor_thresh = lt-20.0; censor_val = -20.0; cnt_thresh = [ >15 ]; cnt_logic = UNION; convert(x) = KM_to_KFT(x); + +FCST_VAR3_NAME = RETOP_L0_ENS_FREQ_ge40 +FCST_VAR3_LEVELS = L0 +FCST_VAR3_THRESH = ==0.1 + +OBS_VAR3_NAME = EchoTop18 +OBS_VAR3_LEVELS = Z500 +OBS_VAR3_THRESH = >=40.0 +OBS_VAR3_OPTIONS = censor_thresh = lt-20.0; censor_val = -20.0; cnt_thresh = [ >15 ]; cnt_logic = UNION; convert(x) = KM_to_KFT(x); + +FCST_VAR4_NAME = RETOP_L0_ENS_FREQ_ge50 +FCST_VAR4_LEVELS = L0 +FCST_VAR4_THRESH = ==0.1 + +OBS_VAR4_NAME = EchoTop18 +OBS_VAR4_LEVELS = Z500 +OBS_VAR4_THRESH = >=50.0 +OBS_VAR4_OPTIONS = censor_thresh = lt-20.0; censor_val = -20.0; cnt_thresh = [ >15 ]; cnt_logic = UNION; convert(x) = KM_to_KFT(x); + +# Neighborhood shape and widths +GRID_STAT_NEIGHBORHOOD_SHAPE = SQUARE +GRID_STAT_NEIGHBORHOOD_WIDTH = 3,5,7 + +# Forecast data description variables +FCST_IS_PROB = True +FCST_PROB_IN_GRIB_PDS = False diff --git a/ush/templates/parm/metplus/PointStat_conus_sfc.conf b/ush/templates/parm/metplus/PointStat_conus_sfc.conf index 1f9e3554f..5fd3f58c9 100644 --- a/ush/templates/parm/metplus/PointStat_conus_sfc.conf +++ b/ush/templates/parm/metplus/PointStat_conus_sfc.conf @@ -1,17 +1,21 @@ +# Grid to Point Example + [dir] # Input and output data directories PB2NC_INPUT_DIR = {ENV[OBS_DIR]} -PB2NC_OUTPUT_DIR = {ENV[EXPTDIR]}/{ENV[CDATE]}/metprd/pb2nc +PB2NC_OUTPUT_DIR = {ENV[EXPTDIR]}/metprd/pb2nc -INPUT_BASE = {ENV[EXPTDIR]}/{ENV[CDATE]}/postprd +INPUT_BASE = {ENV[INPUT_BASE]} FCST_POINT_STAT_INPUT_DIR = {INPUT_BASE} OBS_POINT_STAT_INPUT_DIR = {PB2NC_OUTPUT_DIR} -OUTPUT_BASE = {ENV[EXPTDIR]} +OUTPUT_BASE = {ENV[OUTPUT_BASE]} POINT_STAT_OUTPUT_DIR = {OUTPUT_BASE} -LOG_DIR = {OUTPUT_BASE}/log +LOG_DIR = {ENV[EXPTDIR]}/log + +STAGING_DIR = {OUTPUT_BASE}/stage/conus_sfc [filename_templates] # Template to look for prepbvur input to PB2NC relative to PB2NC_INPUT_DIR @@ -23,7 +27,13 @@ PB2NC_OUTPUT_TEMPLATE = prepbufr.ndas.{valid?fmt=%Y%m%d%H}.nc FCST_POINT_STAT_INPUT_TEMPLATE = {ENV[NET]}.t{init?fmt=%H}z.prslevf{lead?fmt=%HHH}.tm{init?fmt=%H}.grib2 OBS_POINT_STAT_INPUT_TEMPLATE = prepbufr.ndas.{valid?fmt=%Y%m%d%H}.nc -POINT_STAT_OUTPUT_TEMPLATE = {init?fmt=%Y%m%d%H}/metprd/point_stat +POINT_STAT_OUTPUT_TEMPLATE = metprd/point_stat + +# Specify the name of the metplus.log file +LOG_METPLUS = {LOG_DIR}/metplus.log.{ENV[LOG_SUFFIX]}_sfc + +# Specify where the location and name of the final metplus_final.conf +METPLUS_CONF={OUTPUT_BASE}/metprd/point_stat/metplus_final.conus_surface.conf [config] @@ -73,7 +83,7 @@ PB2NC_MESSAGE_TYPE = ADPSFC, ADPUPA # Leave empty to process all # PB2NC_OBS_BUFR_VAR_LIST = POB, QOB, TOB, ZOB, UOB, VOB, PMO, TOCC, TDO, HOVI, D_DPT, D_WDIR, D_WIND, D_RH, D_MIXR, D_PRMSL -PB2NC_OBS_BUFR_VAR_LIST = PMO, ZOB, TOB, TDO, QOB, UOB, VOB, PWO, TOCC, D_RH, HOVI, CEILING, D_PBL, D_CAPE, MXGS +PB2NC_OBS_BUFR_VAR_LIST = PMO, ZOB, TOB, TDO, QOB, UOB, VOB, PWO, TOCC, D_RH, HOVI, CEILING, D_PBL, D_CAPE, MXGS, D_WIND # For defining the time periods for summarization # False for no time summary, True otherwise @@ -135,26 +145,31 @@ BOTH_VAR5_NAME = VGRD BOTH_VAR5_LEVELS = Z10 BOTH_VAR5_THRESH = >=2.572 ;; m/s or 5kts -BOTH_VAR6_NAME = PRMSL -BOTH_VAR6_LEVELS = Z0 +BOTH_VAR6_NAME = WIND +BOTH_VAR6_LEVELS = Z10 +BOTH_VAR6_THRESH = >=2.572 ;; m/s or 5kts +BOTH_VAR6_OPTIONS = GRIB2_pdt = 0; ;; derive instantaneous 10-m wind from U/V components, overriding max 10-m wind -BOTH_VAR7_NAME = TCDC -BOTH_VAR7_LEVELS = L0 -BOTH_VAR7_OPTIONS = GRIB_lvl_typ = 200; interp = { type = [ { method = NEAREST; width = 1; } ]; } +BOTH_VAR7_NAME = PRMSL +BOTH_VAR7_LEVELS = Z0 -BOTH_VAR8_NAME = VIS +BOTH_VAR8_NAME = TCDC BOTH_VAR8_LEVELS = L0 -BOTH_VAR8_THRESH = <805, <1609, <4828, <8045 ,>=8045, <16090 -BOTH_VAR8_OPTIONS = interp = { type = [ { method = NEAREST; width = 1; } ]; } - -BOTH_VAR9_NAME = GUST -BOTH_VAR9_LEVELS = Z0 - -FCST_VAR10_NAME = HGT -FCST_VAR10_LEVELS = L0 -FCST_VAR10_OPTIONS = GRIB_lvl_typ = 215; desc = "CEILING"; -FCST_VAR10_THRESH = <152, <305, <914, <1520, <3040, >=914 -OBS_VAR10_NAME = CEILING -OBS_VAR10_LEVELS = L0 -OBS_VAR10_OPTIONS = GRIB_lvl_typ = 215; interp = { type = [ { method = NEAREST; width = 1; } ]; } -OBS_VAR10_THRESH = <152, <305, <914, <1520, <3040, >=914 +BOTH_VAR8_OPTIONS = GRIB_lvl_typ = 200; interp = { type = [ { method = NEAREST; width = 1; } ]; } + +BOTH_VAR9_NAME = VIS +BOTH_VAR9_LEVELS = L0 +BOTH_VAR9_THRESH = <805, <1609, <4828, <8045 ,>=8045, <16090 +BOTH_VAR9_OPTIONS = interp = { type = [ { method = NEAREST; width = 1; } ]; } + +BOTH_VAR10_NAME = GUST +BOTH_VAR10_LEVELS = Z0 + +FCST_VAR11_NAME = HGT +FCST_VAR11_LEVELS = L0 +FCST_VAR11_OPTIONS = GRIB_lvl_typ = 215; desc = "CEILING"; +FCST_VAR11_THRESH = <152, <305, <914, <1520, <3040, >=914 +OBS_VAR11_NAME = CEILING +OBS_VAR11_LEVELS = L0 +OBS_VAR11_OPTIONS = GRIB_lvl_typ = 215; interp = { type = [ { method = NEAREST; width = 1; } ]; } +OBS_VAR11_THRESH = <152, <305, <914, <1520, <3040, >=914 diff --git a/ush/templates/parm/metplus/PointStat_conus_sfc_mean.conf b/ush/templates/parm/metplus/PointStat_conus_sfc_mean.conf new file mode 100644 index 000000000..c5adc81df --- /dev/null +++ b/ush/templates/parm/metplus/PointStat_conus_sfc_mean.conf @@ -0,0 +1,151 @@ +# Ensemble Mean Grid to Point Example + +[dir] +# Input and output data directories +PB2NC_INPUT_DIR = {ENV[OBS_DIR]} +PB2NC_OUTPUT_DIR = {ENV[EXPTDIR]}/metprd/pb2nc + +INPUT_BASE = {ENV[INPUT_BASE]} +FCST_POINT_STAT_INPUT_DIR = {INPUT_BASE} + +OBS_POINT_STAT_INPUT_DIR = {PB2NC_OUTPUT_DIR} + +OUTPUT_BASE = {ENV[EXPTDIR]} +POINT_STAT_OUTPUT_DIR = {OUTPUT_BASE} + +LOG_DIR = {ENV[EXPTDIR]}/log + +STAGING_DIR = {OUTPUT_BASE}/stage/conus_sfc_mean + +[filename_templates] +# Template to look for prepbvur input to PB2NC relative to PB2NC_INPUT_DIR +PB2NC_INPUT_TEMPLATE = prepbufr.ndas.{valid?fmt=%Y%m%d%H} +# Template to use to write output from PB2NC +PB2NC_OUTPUT_TEMPLATE = prepbufr.ndas.{valid?fmt=%Y%m%d%H}.nc + +# Template to look for forecast/observation input to PointStat relative to FCST_POINT_STAT_INPUT_DIR +FCST_POINT_STAT_INPUT_TEMPLATE = ensemble_stat_{ENV[MODEL]}_ADPSFC_{OBTYPE}_{valid?fmt=%Y%m%d}_{valid?fmt=%H%M%S}V_ens.nc +OBS_POINT_STAT_INPUT_TEMPLATE = prepbufr.ndas.{valid?fmt=%Y%m%d%H}.nc + +POINT_STAT_OUTPUT_TEMPLATE = {init?fmt=%Y%m%d%H}/metprd/ensemble_stat_mean + +# Specify the name of the metplus.log file +LOG_METPLUS = {LOG_DIR}/metplus.log.{ENV[LOG_SUFFIX]}_sfc + +# Specify where the location and name of the final metplus_final.conf +METPLUS_CONF={OUTPUT_BASE}/{ENV[CDATE]}/metprd/ensemble_stat_mean/metplus_final.conus_surface.conf + +[config] +# List of applications to run +PROCESS_LIST = PB2NC, PointStat + +# time looping - options are INIT, VALID, RETRO, and REALTIME +LOOP_BY = INIT + +# Start time for METplus run +INIT_BEG = {ENV[CDATE]} +# End time for METplus run +INIT_END = {ENV[CDATE]} +# Increment between METplus runs in seconds. Must be > 60 +INIT_INCREMENT = 3600 +INIT_TIME_FMT = %Y%m%d%H + +# list of forecast leads to process. +LEAD_SEQ = {ENV[fhr_list]} + +# Order of loops to process data - Options are times, processes +LOOP_ORDER = times + +# Verbosity of MET output - overrides LOG_VERBOSITY for PointStat only +LOG_POINT_STAT_VERBOSITY = 2 + +# For both PB2NC and point_stat +OBS_WINDOW_BEGIN = -1799 +OBS_WINDOW_END = 1800 + +PB2NC_WINDOW_BEGIN = {OBS_WINDOW_BEGIN} +PB2NC_WINDOW_END = {OBS_WINDOW_END} + +# Location of MET config file to pass to PB2NC +PB2NC_CONFIG_FILE = {CONFIG_DIR}/PB2NCConfig + +# If set to True, skip run if the output file determined by the output directory and +# filename template already exists +PB2NC_SKIP_IF_OUTPUT_EXISTS = True + +# Values to pass to pb2nc config file using environment variables of the same name. +PB2NC_GRID = +PB2NC_POLY = +PB2NC_STATION_ID = +PB2NC_MESSAGE_TYPE = + +# Leave empty to process all +# PB2NC_OBS_BUFR_VAR_LIST = POB, QOB, TOB, ZOB, UOB, VOB, PMO, TOCC, TDO, HOVI, D_DPT, D_WDIR, D_WIND, D_RH, D_MIXR, D_PRMSL +PB2NC_OBS_BUFR_VAR_LIST = PMO, ZOB, TOB, TDO, QOB, UOB, VOB, PWO, TOCC, D_RH, HOVI, CEILING, D_PBL, D_CAPE, MXGS, D_WIND + +# For defining the time periods for summarization +# False for no time summary, True otherwise +# The rest of the PB2NC_TIME_SUMMARY variables are ignored if set to False +PB2NC_TIME_SUMMARY_FLAG = False +PB2NC_TIME_SUMMARY_BEG = 000000 +PB2NC_TIME_SUMMARY_END = 235959 +PB2NC_TIME_SUMMARY_VAR_NAMES = PMO,TOB,TDO,UOB,VOB,PWO,TOCC +PB2NC_TIME_SUMMARY_TYPES = min, max, range, mean, stdev, median, p80 + +# Location of MET config file to pass to GridStat +# References PARM_BASE which is the location of the parm directory corresponding +# to the ush directory of the master_metplus.py script that is called +# or the value of the environment variable METPLUS_PARM_BASE if set +POINT_STAT_CONFIG_FILE ={CONFIG_DIR}/PointStatConfig_mean + +# Model/fcst and obs name, e.g. GFS, NAM, GDAS, etc. +MODEL = {ENV[MODEL]}_mean +OBTYPE = NDAS + +# Regrid to specified grid. Indicate NONE if no regridding, or the grid id +POINT_STAT_REGRID_TO_GRID = NONE + +POINT_STAT_OUTPUT_PREFIX = {ENV[MODEL]}_ADPSFC_{OBTYPE}_mean + +# Verification Masking regions +# Indicate which grid and polygon masking region, if applicable +POINT_STAT_GRID = + +# List of full path to poly masking files. NOTE: Only short lists of poly +# files work (those that fit on one line), a long list will result in an +# environment variable that is too long, resulting in an error. For long +# lists of poly masking files (i.e. all the mask files in the NCEP_mask +# directory), define these in the MET point_stat configuration file. +POINT_STAT_POLY = {MET_INSTALL_DIR}/share/met/poly/CONUS.poly +POINT_STAT_STATION_ID = + +# Message types, if all message types are to be returned, leave this empty, +# otherwise indicate the message types of interest. +POINT_STAT_MESSAGE_TYPE = ADPSFC + +# Variables and levels as specified in the field dictionary of the MET +# point_stat configuration file. Specify as FCST_VARn_NAME, FCST_VARn_LEVELS, +# (optional) FCST_VARn_OPTION +FCST_VAR1_NAME = TMP_Z2_ENS_MEAN +FCST_VAR1_LEVELS = Z2 +FCST_VAR1_THRESH = >=293, >=298, >=303 + +OBS_VAR1_NAME = TMP +OBS_VAR1_LEVELS = Z2 +OBS_VAR1_THRESH = >=293, >=298, >=303 + +FCST_VAR2_NAME = DPT_Z2_ENS_MEAN +FCST_VAR2_LEVELS = Z2 +FCST_VAR2_THRESH = >=288, >=293, >=298 + +OBS_VAR2_NAME = DPT +OBS_VAR2_LEVELS = Z2 +OBS_VAR2_THRESH = >=288, >=293, >=298 + +FCST_VAR3_NAME = WIND_Z10_ENS_MEAN +FCST_VAR3_LEVELS = Z10 +FCST_VAR3_THRESH = >=5, >=10 + +OBS_VAR3_NAME = WIND +OBS_VAR3_LEVELS = Z10 +OBS_VAR3_THRESH = >=5, >=10 diff --git a/ush/templates/parm/metplus/PointStat_conus_sfc_prob.conf b/ush/templates/parm/metplus/PointStat_conus_sfc_prob.conf new file mode 100644 index 000000000..f7ef100f7 --- /dev/null +++ b/ush/templates/parm/metplus/PointStat_conus_sfc_prob.conf @@ -0,0 +1,195 @@ +# Ensemble Prob Grid to Point Example + +[dir] +# Input and output data directories +PB2NC_INPUT_DIR = {ENV[OBS_DIR]} +PB2NC_OUTPUT_DIR = {ENV[EXPTDIR]}/metprd/pb2nc + +INPUT_BASE = {ENV[INPUT_BASE]} +FCST_POINT_STAT_INPUT_DIR = {INPUT_BASE} + +OBS_POINT_STAT_INPUT_DIR = {PB2NC_OUTPUT_DIR} + +OUTPUT_BASE = {ENV[EXPTDIR]} +POINT_STAT_OUTPUT_DIR = {OUTPUT_BASE} + +LOG_DIR = {ENV[EXPTDIR]}/log + +STAGING_DIR = {OUTPUT_BASE}/stage/conus_sfc_prob + +[filename_templates] +# Template to look for prepbvur input to PB2NC relative to PB2NC_INPUT_DIR +PB2NC_INPUT_TEMPLATE = prepbufr.ndas.{valid?fmt=%Y%m%d%H} +# Template to use to write output from PB2NC +PB2NC_OUTPUT_TEMPLATE = prepbufr.ndas.{valid?fmt=%Y%m%d%H}.nc + +# Template to look for forecast/observation input to PointStat relative to FCST_POINT_STAT_INPUT_DIR +FCST_POINT_STAT_INPUT_TEMPLATE = ensemble_stat_{ENV[MODEL]}_ADPSFC_{OBTYPE}_{valid?fmt=%Y%m%d}_{valid?fmt=%H%M%S}V_ens.nc +OBS_POINT_STAT_INPUT_TEMPLATE = prepbufr.ndas.{valid?fmt=%Y%m%d%H}.nc + +POINT_STAT_OUTPUT_TEMPLATE = {init?fmt=%Y%m%d%H}/metprd/ensemble_stat_prob + +# Specify the name of the metplus.log file +LOG_METPLUS = {LOG_DIR}/metplus.log.{ENV[LOG_SUFFIX]}_sfc + +# Specify where the location and name of the final metplus_final.conf +METPLUS_CONF={OUTPUT_BASE}/{ENV[CDATE]}/metprd/ensemble_stat_prob/metplus_final.conus_surface.conf + +[config] +# List of applications to run +PROCESS_LIST = PB2NC, PointStat + +# time looping - options are INIT, VALID, RETRO, and REALTIME +LOOP_BY = INIT + +# Start time for METplus run +INIT_BEG = {ENV[CDATE]} +# End time for METplus run +INIT_END = {ENV[CDATE]} +# Increment between METplus runs in seconds. Must be > 60 +INIT_INCREMENT = 3600 +INIT_TIME_FMT = %Y%m%d%H + +# list of forecast leads to process. +LEAD_SEQ = {ENV[fhr_list]} + +# Order of loops to process data - Options are times, processes +LOOP_ORDER = times + +# Verbosity of MET output - overrides LOG_VERBOSITY for PointStat only +LOG_POINT_STAT_VERBOSITY = 2 + +# For both PB2NC and point_stat +OBS_WINDOW_BEGIN = -1799 +OBS_WINDOW_END = 1800 + +PB2NC_WINDOW_BEGIN = {OBS_WINDOW_BEGIN} +PB2NC_WINDOW_END = {OBS_WINDOW_END} + +# Location of MET config file to pass to PB2NC +PB2NC_CONFIG_FILE = {CONFIG_DIR}/PB2NCConfig + +# If set to True, skip run if the output file determined by the output directory and +# filename template already exists +PB2NC_SKIP_IF_OUTPUT_EXISTS = True + +# Values to pass to pb2nc config file using environment variables of the same name. +PB2NC_GRID = +PB2NC_POLY = +PB2NC_STATION_ID = +PB2NC_MESSAGE_TYPE = + +# Leave empty to process all +# PB2NC_OBS_BUFR_VAR_LIST = POB, QOB, TOB, ZOB, UOB, VOB, PMO, TOCC, TDO, HOVI, D_DPT, D_WDIR, D_WIND, D_RH, D_MIXR, D_PRMSL +PB2NC_OBS_BUFR_VAR_LIST = PMO, ZOB, TOB, TDO, QOB, UOB, VOB, PWO, TOCC, D_RH, HOVI, CEILING, D_PBL, D_CAPE, MXGS, D_WIND + +# For defining the time periods for summarization +# False for no time summary, True otherwise +# The rest of the PB2NC_TIME_SUMMARY variables are ignored if set to False +PB2NC_TIME_SUMMARY_FLAG = False +PB2NC_TIME_SUMMARY_BEG = 000000 +PB2NC_TIME_SUMMARY_END = 235959 +PB2NC_TIME_SUMMARY_VAR_NAMES = PMO,TOB,TDO,UOB,VOB,PWO,TOCC +PB2NC_TIME_SUMMARY_TYPES = min, max, range, mean, stdev, median, p80 + +# Location of MET config file to pass to GridStat +# References PARM_BASE which is the location of the parm directory corresponding +# to the ush directory of the master_metplus.py script that is called +# or the value of the environment variable METPLUS_PARM_BASE if set +POINT_STAT_CONFIG_FILE ={CONFIG_DIR}/PointStatConfig_prob + +# Model/fcst and obs name, e.g. GFS, NAM, GDAS, etc. +MODEL = {ENV[MODEL]}_prob +OBTYPE = NDAS + +# Regrid to specified grid. Indicate NONE if no regridding, or the grid id +POINT_STAT_REGRID_TO_GRID = NONE + +POINT_STAT_OUTPUT_PREFIX = {ENV[MODEL]}_ADPSFC_{OBTYPE}_prob + +# Verification Masking regions +# Indicate which grid and polygon masking region, if applicable +POINT_STAT_GRID = + +# List of full path to poly masking files. NOTE: Only short lists of poly +# files work (those that fit on one line), a long list will result in an +# environment variable that is too long, resulting in an error. For long +# lists of poly masking files (i.e. all the mask files in the NCEP_mask +# directory), define these in the MET point_stat configuration file. +POINT_STAT_POLY = {MET_INSTALL_DIR}/share/met/poly/CONUS.poly +POINT_STAT_STATION_ID = + +# Message types, if all message types are to be returned, leave this empty, +# otherwise indicate the message types of interest. +POINT_STAT_MESSAGE_TYPE = ADPSFC + +# Variables and levels as specified in the field dictionary of the MET +# point_stat configuration file. Specify as FCST_VARn_NAME, FCST_VARn_LEVELS, +# (optional) FCST_VARn_OPTION +FCST_VAR1_NAME = TMP_Z2_ENS_FREQ_ge293 +FCST_VAR1_LEVELS = (*,*) +FCST_VAR1_THRESH = ==0.1 + +OBS_VAR1_NAME = TMP +OBS_VAR1_LEVELS = Z2 +OBS_VAR1_THRESH = >=293 + +FCST_VAR2_NAME = TMP_Z2_ENS_FREQ_ge298 +FCST_VAR2_LEVELS = (*,*) +FCST_VAR2_THRESH = ==0.1 + +OBS_VAR2_NAME = TMP +OBS_VAR2_LEVELS = Z2 +OBS_VAR2_THRESH = >=298 + +FCST_VAR3_NAME = TMP_Z2_ENS_FREQ_ge303 +FCST_VAR3_LEVELS = (*,*) +FCST_VAR3_THRESH = ==0.1 + +OBS_VAR3_NAME = TMP +OBS_VAR3_LEVELS = Z2 +OBS_VAR3_THRESH = >=303 + +FCST_VAR4_NAME = DPT_Z2_ENS_FREQ_ge288 +FCST_VAR4_LEVELS = (*,*) +FCST_VAR4_THRESH = ==0.1 + +OBS_VAR4_NAME = DPT +OBS_VAR4_LEVELS = Z2 +OBS_VAR4_THRESH = >=288 + +FCST_VAR5_NAME = DPT_Z2_ENS_FREQ_ge293 +FCST_VAR5_LEVELS = (*,*) +FCST_VAR5_THRESH = ==0.1 + +OBS_VAR5_NAME = DPT +OBS_VAR5_LEVELS = Z2 +OBS_VAR5_THRESH = >=293 + +FCST_VAR6_NAME = DPT_Z2_ENS_FREQ_ge298 +FCST_VAR6_LEVELS = (*,*) +FCST_VAR6_THRESH = ==0.1 + +OBS_VAR6_NAME = DPT +OBS_VAR6_LEVELS = Z2 +OBS_VAR6_THRESH = >=298 + +FCST_VAR7_NAME = WIND_Z10_ENS_FREQ_ge5 +FCST_VAR7_LEVELS = Z10 +FCST_VAR7_THRESH = ==0.1 + +OBS_VAR7_NAME = WIND +OBS_VAR7_LEVELS = Z10 +OBS_VAR7_THRESH = >=5 + +FCST_VAR8_NAME = WIND_Z10_ENS_FREQ_ge10 +FCST_VAR8_LEVELS = Z10 +FCST_VAR8_THRESH = ==0.1 + +OBS_VAR8_NAME = WIND +OBS_VAR8_LEVELS = Z10 +OBS_VAR8_THRESH = >=10 + +# Forecast data description variables +FCST_IS_PROB = True +FCST_PROB_IN_GRIB_PDS = False diff --git a/ush/templates/parm/metplus/PointStat_upper_air.conf b/ush/templates/parm/metplus/PointStat_upper_air.conf index f9d3083ca..cd05d930b 100644 --- a/ush/templates/parm/metplus/PointStat_upper_air.conf +++ b/ush/templates/parm/metplus/PointStat_upper_air.conf @@ -1,21 +1,26 @@ +# Grid to Point Example + [dir] # Input and output data directories PB2NC_INPUT_DIR = {ENV[OBS_DIR]} -PB2NC_OUTPUT_DIR = {ENV[EXPTDIR]}/{ENV[CDATE]}/metprd/pb2nc +PB2NC_OUTPUT_DIR = {ENV[EXPTDIR]}/metprd/pb2nc -INPUT_BASE = {ENV[EXPTDIR]}/{ENV[CDATE]}/postprd +INPUT_BASE = {ENV[INPUT_BASE]} FCST_POINT_STAT_INPUT_DIR = {INPUT_BASE} OBS_POINT_STAT_INPUT_DIR = {PB2NC_OUTPUT_DIR} -OUTPUT_BASE = {ENV[EXPTDIR]} +OUTPUT_BASE = {ENV[OUTPUT_BASE]} POINT_STAT_OUTPUT_DIR = {OUTPUT_BASE} -LOG_DIR = {OUTPUT_BASE}/log +LOG_DIR = {ENV[EXPTDIR]}/log + +STAGING_DIR = {OUTPUT_BASE}/stage/upper_air [filename_templates] # Template to look for prepbvur input to PB2NC relative to PB2NC_INPUT_DIR -PB2NC_INPUT_TEMPLATE = prepbufr.ndas.{valid?fmt=%Y%m%d%H} +PB2NC_INPUT_TEMPLATE = prepbufr.ndas.{valid?fmt=%Y%m%d%H} + # Template to use to write output from PB2NC PB2NC_OUTPUT_TEMPLATE = prepbufr.ndas.{valid?fmt=%Y%m%d%H}.nc @@ -23,10 +28,15 @@ PB2NC_OUTPUT_TEMPLATE = prepbufr.ndas.{valid?fmt=%Y%m%d%H}.nc FCST_POINT_STAT_INPUT_TEMPLATE = {ENV[NET]}.t{init?fmt=%H}z.prslevf{lead?fmt=%HHH}.tm{init?fmt=%H}.grib2 OBS_POINT_STAT_INPUT_TEMPLATE = prepbufr.ndas.{valid?fmt=%Y%m%d%H}.nc -POINT_STAT_OUTPUT_TEMPLATE = {init?fmt=%Y%m%d%H}/metprd/point_stat +POINT_STAT_OUTPUT_TEMPLATE = metprd/point_stat -[config] +# Specify the name of the metplus.log file +LOG_METPLUS = {LOG_DIR}/metplus.log.{ENV[LOG_SUFFIX]}_upa + +# Specify where the location and name of the final metplus_final.conf +METPLUS_CONF={OUTPUT_BASE}/metprd/point_stat/metplus_final.upper_air.conf +[config] # List of applications to run PROCESS_LIST = PB2NC, PointStat @@ -73,7 +83,7 @@ PB2NC_MESSAGE_TYPE = ADPSFC, ADPUPA # Leave empty to process all # PB2NC_OBS_BUFR_VAR_LIST = POB, QOB, TOB, ZOB, UOB, VOB, PMO, TOCC, TDO, HOVI, D_DPT, D_WDIR, D_WIND, D_RH, D_MIXR, D_PRMSL -PB2NC_OBS_BUFR_VAR_LIST = PMO, ZOB, TOB, TDO, QOB, UOB, VOB, PWO, TOCC, D_RH, HOVI, CEILING, D_PBL, D_CAPE, MXGS +PB2NC_OBS_BUFR_VAR_LIST = PMO, ZOB, TOB, TDO, QOB, UOB, VOB, PWO, TOCC, D_RH, HOVI, CEILING, D_PBL, D_CAPE, MXGS, D_WIND # For defining the time periods for summarization # False for no time summary, True otherwise @@ -135,30 +145,34 @@ BOTH_VAR5_NAME = VGRD BOTH_VAR5_LEVELS = P1000, P925, P850, P700, P500, P400, P300, P250, P200, P150, P100, P50, P20, P10 BOTH_VAR5_THRESH = >=2.572 ;; m/s or 5kts -BOTH_VAR6_NAME = HGT -BOTH_VAR6_LEVELS = P1000, P950, P925, P850, P700, P500, P400, P300, P250, P200, P150, P100, P50, P20, P10 - -BOTH_VAR7_NAME = SPFH -BOTH_VAR7_LEVELS = P1000, P850, P700, P500, P400, P300 - -FCST_VAR8_NAME = CAPE -FCST_VAR8_LEVELS = L0 -FCST_VAR8_OPTIONS = cnt_thresh = [ >0 ]; -FCST_VAR8_THRESH = >500, >1000, >1500, >2000, >3000, >4000 -OBS_VAR8_NAME = CAPE -OBS_VAR8_LEVELS = L0-100000 -OBS_VAR8_OPTIONS = cnt_thresh = [ >0 ]; cnt_logic = UNION; -OBS_VAR8_THRESH = >500, >1000, >1500, >2000, >3000, >4000 - -FCST_VAR9_NAME = HPBL -FCST_VAR9_LEVELS = Z0 -OBS_VAR9_NAME = PBL -OBS_VAR9_LEVELS = L0 -OBS_VAR9_OPTIONS = desc = "TKE"; - -FCST_VAR10_NAME = HGT -FCST_VAR10_LEVELS = L0 -FCST_VAR10_OPTIONS = GRIB_lvl_typ = 220; +BOTH_VAR6_NAME = WIND +BOTH_VAR6_LEVELS = P1000, P925, P850, P700, P500, P400, P300, P250, P200, P150, P100, P50, P20, P10 +BOTH_VAR6_THRESH = >=2.572 ;; m/s or 5kts + +BOTH_VAR7_NAME = HGT +BOTH_VAR7_LEVELS = P1000, P950, P925, P850, P700, P500, P400, P300, P250, P200, P150, P100, P50, P20, P10 + +BOTH_VAR8_NAME = SPFH +BOTH_VAR8_LEVELS = P1000, P850, P700, P500, P400, P300 + +FCST_VAR9_NAME = CAPE +FCST_VAR9_LEVELS = L0 +FCST_VAR9_OPTIONS = cnt_thresh = [ >0 ]; +FCST_VAR9_THRESH = >500, >1000, >1500, >2000, >3000, >4000 +OBS_VAR9_NAME = CAPE +OBS_VAR9_LEVELS = L0-100000 +OBS_VAR9_OPTIONS = cnt_thresh = [ >0 ]; cnt_logic = UNION; +OBS_VAR9_THRESH = >500, >1000, >1500, >2000, >3000, >4000 + +FCST_VAR10_NAME = HPBL +FCST_VAR10_LEVELS = Z0 OBS_VAR10_NAME = PBL OBS_VAR10_LEVELS = L0 -OBS_VAR10_OPTIONS = desc = "RI"; +OBS_VAR10_OPTIONS = desc = "TKE"; + +FCST_VAR11_NAME = HGT +FCST_VAR11_LEVELS = L0 +FCST_VAR11_OPTIONS = GRIB_lvl_typ = 220; +OBS_VAR11_NAME = PBL +OBS_VAR11_LEVELS = L0 +OBS_VAR11_OPTIONS = desc = "RI"; diff --git a/ush/templates/parm/metplus/PointStat_upper_air_mean.conf b/ush/templates/parm/metplus/PointStat_upper_air_mean.conf new file mode 100644 index 000000000..e6a1a062a --- /dev/null +++ b/ush/templates/parm/metplus/PointStat_upper_air_mean.conf @@ -0,0 +1,152 @@ +# Ensemble Mean Grid to Point Example + +[dir] +# Input and output data directories +PB2NC_INPUT_DIR = {ENV[OBS_DIR]} +PB2NC_OUTPUT_DIR = {ENV[EXPTDIR]}/metprd/pb2nc + +INPUT_BASE = {ENV[INPUT_BASE]} +FCST_POINT_STAT_INPUT_DIR = {INPUT_BASE} + +OBS_POINT_STAT_INPUT_DIR = {PB2NC_OUTPUT_DIR} + +OUTPUT_BASE = {ENV[EXPTDIR]} +POINT_STAT_OUTPUT_DIR = {OUTPUT_BASE} + +LOG_DIR = {ENV[EXPTDIR]}/log + +STAGING_DIR = {OUTPUT_BASE}/stage/upper_air_mean + +[filename_templates] +# Template to look for prepbvur input to PB2NC relative to PB2NC_INPUT_DIR +PB2NC_INPUT_TEMPLATE = prepbufr.ndas.{valid?fmt=%Y%m%d%H} +# Template to use to write output from PB2NC +PB2NC_OUTPUT_TEMPLATE = prepbufr.ndas.{valid?fmt=%Y%m%d%H}.nc + +# Template to look for forecast/observation input to PointStat relative to FCST_POINT_STAT_INPUT_DIR +FCST_POINT_STAT_INPUT_TEMPLATE = ensemble_stat_{ENV[MODEL]}_ADPUPA_{OBTYPE}_{valid?fmt=%Y%m%d}_{valid?fmt=%H%M%S}V_ens.nc +OBS_POINT_STAT_INPUT_TEMPLATE = prepbufr.ndas.{valid?fmt=%Y%m%d%H}.nc + +POINT_STAT_OUTPUT_TEMPLATE = {init?fmt=%Y%m%d%H}/metprd/ensemble_stat_mean + +# Specify the name of the metplus.log file +LOG_METPLUS = {LOG_DIR}/metplus.log.{ENV[LOG_SUFFIX]}_upa + +# Specify where the location and name of the final metplus_final.conf +METPLUS_CONF={OUTPUT_BASE}/{ENV[CDATE]}/metprd/ensemble_stat_mean/metplus_final.upper_air.conf + +[config] +# List of applications to run +PROCESS_LIST = PB2NC, PointStat + +# time looping - options are INIT, VALID, RETRO, and REALTIME +LOOP_BY = INIT + +# Start time for METplus run +INIT_BEG = {ENV[CDATE]} +# End time for METplus run +INIT_END = {ENV[CDATE]} +# Increment between METplus runs in seconds. Must be > 60 +INIT_INCREMENT = 3600 +INIT_TIME_FMT = %Y%m%d%H + +# list of forecast leads to process. +LEAD_SEQ = begin_end_incr(0,{ENV[fhr_last]},6) +#LEAD_SEQ = {ENV[fhr_list]} + +# Order of loops to process data - Options are times, processes +LOOP_ORDER = times + +# Verbosity of MET output - overrides LOG_VERBOSITY for PointStat only +LOG_POINT_STAT_VERBOSITY = 2 + +# For both PB2NC and point_stat +OBS_WINDOW_BEGIN = -1799 +OBS_WINDOW_END = 1800 + +PB2NC_WINDOW_BEGIN = {OBS_WINDOW_BEGIN} +PB2NC_WINDOW_END = {OBS_WINDOW_END} + +# Location of MET config file to pass to PB2NC +PB2NC_CONFIG_FILE = {CONFIG_DIR}/PB2NCConfig + +# If set to True, skip run if the output file determined by the output directory and +# filename template already exists +PB2NC_SKIP_IF_OUTPUT_EXISTS = True + +# Values to pass to pb2nc config file using environment variables of the same name. +PB2NC_GRID = +PB2NC_POLY = +PB2NC_STATION_ID = +PB2NC_MESSAGE_TYPE = + +# Leave empty to process all +# PB2NC_OBS_BUFR_VAR_LIST = POB, QOB, TOB, ZOB, UOB, VOB, PMO, TOCC, TDO, HOVI, D_DPT, D_WDIR, D_WIND, D_RH, D_MIXR, D_PRMSL +PB2NC_OBS_BUFR_VAR_LIST = PMO, ZOB, TOB, TDO, QOB, UOB, VOB, PWO, TOCC, D_RH, HOVI, CEILING, D_PBL, D_CAPE, MXGS, D_WIND + +# For defining the time periods for summarization +# False for no time summary, True otherwise +# The rest of the PB2NC_TIME_SUMMARY variables are ignored if set to False +PB2NC_TIME_SUMMARY_FLAG = False +PB2NC_TIME_SUMMARY_BEG = 000000 +PB2NC_TIME_SUMMARY_END = 235959 +PB2NC_TIME_SUMMARY_VAR_NAMES = PMO,TOB,TDO,UOB,VOB,PWO,TOCC +PB2NC_TIME_SUMMARY_TYPES = min, max, range, mean, stdev, median, p80 + +# Location of MET config file to pass to GridStat +# References PARM_BASE which is the location of the parm directory corresponding +# to the ush directory of the master_metplus.py script that is called +# or the value of the environment variable METPLUS_PARM_BASE if set +POINT_STAT_CONFIG_FILE ={CONFIG_DIR}/PointStatConfig_mean + +# Model/fcst and obs name, e.g. GFS, NAM, GDAS, etc. +MODEL = {ENV[MODEL]}_mean +OBTYPE = NDAS + +# Regrid to specified grid. Indicate NONE if no regridding, or the grid id +POINT_STAT_REGRID_TO_GRID = NONE + +POINT_STAT_OUTPUT_PREFIX = {ENV[MODEL]}_ADPUPA_{OBTYPE}_mean + +# Verification Masking regions +# Indicate which grid and polygon masking region, if applicable +POINT_STAT_GRID = + +# List of full path to poly masking files. NOTE: Only short lists of poly +# files work (those that fit on one line), a long list will result in an +# environment variable that is too long, resulting in an error. For long +# lists of poly masking files (i.e. all the mask files in the NCEP_mask +# directory), define these in the MET point_stat configuration file. +POINT_STAT_POLY = {MET_INSTALL_DIR}/share/met/poly/CONUS.poly +POINT_STAT_STATION_ID = + +# Message types, if all message types are to be returned, leave this empty, +# otherwise indicate the message types of interest. +POINT_STAT_MESSAGE_TYPE = ADPUPA + +# Variables and levels as specified in the field dictionary of the MET +# point_stat configuration file. Specify as FCST_VARn_NAME, FCST_VARn_LEVELS, +# (optional) FCST_VARn_OPTION +FCST_VAR1_NAME = TMP_P850_ENS_MEAN +FCST_VAR1_LEVELS = P850 +FCST_VAR1_THRESH = >=288, >=293, >=298 + +OBS_VAR1_NAME = TMP +OBS_VAR1_LEVELS = P850 +OBS_VAR1_THRESH = >=288, >=293, >=298 + +FCST_VAR2_NAME = DPT_P850_ENS_MEAN +FCST_VAR2_LEVELS = P850 +FCST_VAR2_THRESH = >=283, >=288, >=293 + +OBS_VAR2_NAME = DPT +OBS_VAR2_LEVELS = P850 +OBS_VAR2_THRESH = >=283, >=288, >=293 + +FCST_VAR3_NAME = WIND_P850_ENS_MEAN +FCST_VAR3_LEVELS = P850 +FCST_VAR3_THRESH = >=5, >=10 + +OBS_VAR3_NAME = WIND +OBS_VAR3_LEVELS = P850 +OBS_VAR3_THRESH = >=5, >=10 diff --git a/ush/templates/parm/metplus/PointStat_upper_air_prob.conf b/ush/templates/parm/metplus/PointStat_upper_air_prob.conf new file mode 100644 index 000000000..bb3790689 --- /dev/null +++ b/ush/templates/parm/metplus/PointStat_upper_air_prob.conf @@ -0,0 +1,196 @@ +# Ensemble Prob Grid to Point Example + +[dir] +# Input and output data directories +PB2NC_INPUT_DIR = {ENV[OBS_DIR]} +PB2NC_OUTPUT_DIR = {ENV[EXPTDIR]}/metprd/pb2nc + +INPUT_BASE = {ENV[INPUT_BASE]} +FCST_POINT_STAT_INPUT_DIR = {INPUT_BASE} + +OBS_POINT_STAT_INPUT_DIR = {PB2NC_OUTPUT_DIR} + +OUTPUT_BASE = {ENV[EXPTDIR]} +POINT_STAT_OUTPUT_DIR = {OUTPUT_BASE} + +LOG_DIR = {ENV[EXPTDIR]}/log + +STAGING_DIR = {OUTPUT_BASE}/stage/upper_air_prob + +[filename_templates] +# Template to look for prepbvur input to PB2NC relative to PB2NC_INPUT_DIR +PB2NC_INPUT_TEMPLATE = prepbufr.ndas.{valid?fmt=%Y%m%d%H} +# Template to use to write output from PB2NC +PB2NC_OUTPUT_TEMPLATE = prepbufr.ndas.{valid?fmt=%Y%m%d%H}.nc + +# Template to look for forecast/observation input to PointStat relative to FCST_POINT_STAT_INPUT_DIR +FCST_POINT_STAT_INPUT_TEMPLATE = ensemble_stat_{ENV[MODEL]}_ADPUPA_{OBTYPE}_{valid?fmt=%Y%m%d}_{valid?fmt=%H%M%S}V_ens.nc +OBS_POINT_STAT_INPUT_TEMPLATE = prepbufr.ndas.{valid?fmt=%Y%m%d%H}.nc + +POINT_STAT_OUTPUT_TEMPLATE = {init?fmt=%Y%m%d%H}/metprd/ensemble_stat_prob + +# Specify the name of the metplus.log file +LOG_METPLUS = {LOG_DIR}/metplus.log.{ENV[LOG_SUFFIX]}_upa + +# Specify where the location and name of the final metplus_final.conf +METPLUS_CONF={OUTPUT_BASE}/{ENV[CDATE]}/metprd/ensemble_stat_prob/metplus_final.upper_air.conf + +[config] +# List of applications to run +PROCESS_LIST = PB2NC, PointStat + +# time looping - options are INIT, VALID, RETRO, and REALTIME +LOOP_BY = INIT + +# Start time for METplus run +INIT_BEG = {ENV[CDATE]} +# End time for METplus run +INIT_END = {ENV[CDATE]} +# Increment between METplus runs in seconds. Must be > 60 +INIT_INCREMENT = 3600 +INIT_TIME_FMT = %Y%m%d%H + +# list of forecast leads to process. +LEAD_SEQ = begin_end_incr(0,{ENV[fhr_last]},6) +#LEAD_SEQ = {ENV[fhr_list]} + +# Order of loops to process data - Options are times, processes +LOOP_ORDER = times + +# Verbosity of MET output - overrides LOG_VERBOSITY for PointStat only +LOG_POINT_STAT_VERBOSITY = 2 + +# For both PB2NC and point_stat +OBS_WINDOW_BEGIN = -1799 +OBS_WINDOW_END = 1800 + +PB2NC_WINDOW_BEGIN = {OBS_WINDOW_BEGIN} +PB2NC_WINDOW_END = {OBS_WINDOW_END} + +# Location of MET config file to pass to PB2NC +PB2NC_CONFIG_FILE = {CONFIG_DIR}/PB2NCConfig + +# If set to True, skip run if the output file determined by the output directory and +# filename template already exists +PB2NC_SKIP_IF_OUTPUT_EXISTS = True + +# Values to pass to pb2nc config file using environment variables of the same name. +PB2NC_GRID = +PB2NC_POLY = +PB2NC_STATION_ID = +PB2NC_MESSAGE_TYPE = + +# Leave empty to process all +# PB2NC_OBS_BUFR_VAR_LIST = POB, QOB, TOB, ZOB, UOB, VOB, PMO, TOCC, TDO, HOVI, D_DPT, D_WDIR, D_WIND, D_RH, D_MIXR, D_PRMSL +PB2NC_OBS_BUFR_VAR_LIST = PMO, ZOB, TOB, TDO, QOB, UOB, VOB, PWO, TOCC, D_RH, HOVI, CEILING, D_PBL, D_CAPE, MXGS, D_WIND + +# For defining the time periods for summarization +# False for no time summary, True otherwise +# The rest of the PB2NC_TIME_SUMMARY variables are ignored if set to False +PB2NC_TIME_SUMMARY_FLAG = False +PB2NC_TIME_SUMMARY_BEG = 000000 +PB2NC_TIME_SUMMARY_END = 235959 +PB2NC_TIME_SUMMARY_VAR_NAMES = PMO,TOB,TDO,UOB,VOB,PWO,TOCC +PB2NC_TIME_SUMMARY_TYPES = min, max, range, mean, stdev, median, p80 + +# Location of MET config file to pass to GridStat +# References PARM_BASE which is the location of the parm directory corresponding +# to the ush directory of the master_metplus.py script that is called +# or the value of the environment variable METPLUS_PARM_BASE if set +POINT_STAT_CONFIG_FILE ={CONFIG_DIR}/PointStatConfig_prob + +# Model/fcst and obs name, e.g. GFS, NAM, GDAS, etc. +MODEL = {ENV[MODEL]}_prob +OBTYPE = NDAS + +# Regrid to specified grid. Indicate NONE if no regridding, or the grid id +POINT_STAT_REGRID_TO_GRID = NONE + +POINT_STAT_OUTPUT_PREFIX = {ENV[MODEL]}_ADPUPA_{OBTYPE}_prob + +# Verification Masking regions +# Indicate which grid and polygon masking region, if applicable +POINT_STAT_GRID = + +# List of full path to poly masking files. NOTE: Only short lists of poly +# files work (those that fit on one line), a long list will result in an +# environment variable that is too long, resulting in an error. For long +# lists of poly masking files (i.e. all the mask files in the NCEP_mask +# directory), define these in the MET point_stat configuration file. +POINT_STAT_POLY = {MET_INSTALL_DIR}/share/met/poly/CONUS.poly +POINT_STAT_STATION_ID = + +# Message types, if all message types are to be returned, leave this empty, +# otherwise indicate the message types of interest. +POINT_STAT_MESSAGE_TYPE = ADPUPA + +# Variables and levels as specified in the field dictionary of the MET +# point_stat configuration file. Specify as FCST_VARn_NAME, FCST_VARn_LEVELS, +# (optional) FCST_VARn_OPTION +FCST_VAR1_NAME = TMP_P850_ENS_FREQ_ge288 +FCST_VAR1_LEVELS = (*,*) +FCST_VAR1_THRESH = ==0.1 + +OBS_VAR1_NAME = TMP +OBS_VAR1_LEVELS = P850 +OBS_VAR1_THRESH = >=288 + +FCST_VAR2_NAME = TMP_P850_ENS_FREQ_ge293 +FCST_VAR2_LEVELS = (*,*) +FCST_VAR2_THRESH = ==0.1 + +OBS_VAR2_NAME = TMP +OBS_VAR2_LEVELS = P850 +OBS_VAR2_THRESH = >=293 + +FCST_VAR3_NAME = TMP_P850_ENS_FREQ_ge298 +FCST_VAR3_LEVELS = (*,*) +FCST_VAR3_THRESH = ==0.1 + +OBS_VAR3_NAME = TMP +OBS_VAR3_LEVELS = P850 +OBS_VAR3_THRESH = >=298 + +FCST_VAR4_NAME = DPT_P850_ENS_FREQ_ge283 +FCST_VAR4_LEVELS = (*,*) +FCST_VAR4_THRESH = ==0.1 + +OBS_VAR4_NAME = DPT +OBS_VAR4_LEVELS = P850 +OBS_VAR4_THRESH = >=283 + +FCST_VAR5_NAME = DPT_P850_ENS_FREQ_ge288 +FCST_VAR5_LEVELS = (*,*) +FCST_VAR5_THRESH = ==0.1 + +OBS_VAR5_NAME = DPT +OBS_VAR5_LEVELS = P850 +OBS_VAR5_THRESH = >=288 + +FCST_VAR6_NAME = DPT_P850_ENS_FREQ_ge293 +FCST_VAR6_LEVELS = (*,*) +FCST_VAR6_THRESH = ==0.1 + +OBS_VAR6_NAME = DPT +OBS_VAR6_LEVELS = P850 +OBS_VAR6_THRESH = >=293 + +FCST_VAR7_NAME = WIND_P850_ENS_FREQ_ge5 +FCST_VAR7_LEVELS = (*,*) +FCST_VAR7_THRESH = ==0.1 + +OBS_VAR7_NAME = WIND +OBS_VAR7_LEVELS = P850 +OBS_VAR7_THRESH = >=5 + +FCST_VAR8_NAME = WIND_P850_ENS_FREQ_ge10 +FCST_VAR8_LEVELS = (*,*) +FCST_VAR8_THRESH = ==0.1 + +OBS_VAR8_NAME = WIND +OBS_VAR8_LEVELS = P850 +OBS_VAR8_THRESH = >=10 + +# Forecast data description variables +FCST_IS_PROB = True +FCST_PROB_IN_GRIB_PDS = False diff --git a/ush/templates/parm/metplus/REFC.conf b/ush/templates/parm/metplus/REFC.conf index 6aa8d198f..9a3fdb2d1 100644 --- a/ush/templates/parm/metplus/REFC.conf +++ b/ush/templates/parm/metplus/REFC.conf @@ -1,17 +1,19 @@ -# Composite Reflectivity Verification +# Composite Reflectivity Verification Options [dir] # Input and output data directories -INPUT_BASE = {ENV[EXPTDIR]}/{ENV[CDATE]}/postprd +INPUT_BASE = {ENV[INPUT_BASE]} FCST_GRID_STAT_INPUT_DIR = {INPUT_BASE} -OUTPUT_BASE = {ENV[EXPTDIR]} +OUTPUT_BASE = {ENV[OUTPUT_BASE]} GRID_STAT_OUTPUT_DIR = {OUTPUT_BASE} -LOG_DIR = {OUTPUT_BASE}/log +LOG_DIR = {ENV[EXPTDIR]}/log OBS_GRID_STAT_INPUT_DIR = {ENV[OBS_DIR]} +STAGING_DIR = {OUTPUT_BASE}/stage/REFC + [filename_templates] # format of filenames FCST_GRID_STAT_INPUT_TEMPLATE = {ENV[NET]}.t{init?fmt=%H}z.prslevf{lead?fmt=%HHH}.tm{init?fmt=%H}.grib2 @@ -19,11 +21,17 @@ FCST_GRID_STAT_INPUT_TEMPLATE = {ENV[NET]}.t{init?fmt=%H}z.prslevf{lead?fmt=%HHH # ANLYS OBS_GRID_STAT_INPUT_TEMPLATE = {valid?fmt=%Y%m%d}/MergedReflectivityQCComposite_00.50_{valid?fmt=%Y%m%d}-{valid?fmt=%H}0000.grib2 -GRID_STAT_OUTPUT_TEMPLATE = {init?fmt=%Y%m%d%H}/metprd/grid_stat +GRID_STAT_OUTPUT_TEMPLATE = metprd/grid_stat # Location of MET poly files GRID_STAT_VERIFICATION_MASK_TEMPLATE = {MET_INSTALL_DIR}/share/met/poly/CONUS.poly +# Specify the name of the metplus.log file +LOG_METPLUS = {LOG_DIR}/metplus.log.{ENV[LOG_SUFFIX]} + +# Specify where the location and name of the final metplus_final.conf +METPLUS_CONF={OUTPUT_BASE}/metprd/grid_stat/metplus_final.REFC.conf + [config] # Model to verify MODEL = {ENV[MODEL]} @@ -37,7 +45,7 @@ OBS_NATIVE_DATA_TYPE = GRIB PROCESS_LIST = GridStat # location of grid_stat MET config file -GRID_STAT_CONFIG_FILE = {CONFIG_DIR}/GridStatConfig_REFL +GRID_STAT_CONFIG_FILE = {CONFIG_DIR}/GridStatConfig_REFC GRID_STAT_OUTPUT_PREFIX = {MODEL}_{CURRENT_FCST_NAME}_{OBTYPE} # time looping - options are INIT, VALID, RETRO, and REALTIME diff --git a/ush/templates/parm/metplus/RETOP.conf b/ush/templates/parm/metplus/RETOP.conf index 5b57d3c34..5508bd600 100644 --- a/ush/templates/parm/metplus/RETOP.conf +++ b/ush/templates/parm/metplus/RETOP.conf @@ -1,17 +1,19 @@ -# Composite Reflectivity Verification +# Echo Top Verification Options [dir] # Input and output data directories -INPUT_BASE = {ENV[EXPTDIR]}/{ENV[CDATE]}/postprd +INPUT_BASE = {ENV[INPUT_BASE]} FCST_GRID_STAT_INPUT_DIR = {INPUT_BASE} -OUTPUT_BASE = {ENV[EXPTDIR]} +OUTPUT_BASE = {ENV[OUTPUT_BASE]} GRID_STAT_OUTPUT_DIR = {OUTPUT_BASE} -LOG_DIR = {OUTPUT_BASE}/log +LOG_DIR = {ENV[EXPTDIR]}/log OBS_GRID_STAT_INPUT_DIR = {ENV[OBS_DIR]} +STAGING_DIR = {OUTPUT_BASE}/stage/RETOP + [filename_templates] # format of filenames FCST_GRID_STAT_INPUT_TEMPLATE = {ENV[NET]}.t{init?fmt=%H}z.prslevf{lead?fmt=%HHH}.tm{init?fmt=%H}.grib2 @@ -19,11 +21,17 @@ FCST_GRID_STAT_INPUT_TEMPLATE = {ENV[NET]}.t{init?fmt=%H}z.prslevf{lead?fmt=%HHH # ANLYS OBS_GRID_STAT_INPUT_TEMPLATE = {valid?fmt=%Y%m%d}/EchoTop_18_00.50_{valid?fmt=%Y%m%d}-{valid?fmt=%H%M%S}.grib2 -GRID_STAT_OUTPUT_TEMPLATE = {init?fmt=%Y%m%d%H}/metprd/grid_stat +GRID_STAT_OUTPUT_TEMPLATE = metprd/grid_stat # Location of MET poly files GRID_STAT_VERIFICATION_MASK_TEMPLATE = {MET_INSTALL_DIR}/share/met/poly/CONUS.poly +# Specify the name of the metplus.log file +LOG_METPLUS = {LOG_DIR}/metplus.log.{ENV[LOG_SUFFIX]} + +# Specify where the location and name of the final metplus_final.conf +METPLUS_CONF={OUTPUT_BASE}/metprd/grid_stat/metplus_final.RETOP.conf + [config] # Model to verify MODEL = {ENV[MODEL]} @@ -37,7 +45,7 @@ OBS_NATIVE_DATA_TYPE = GRIB PROCESS_LIST = GridStat # location of grid_stat MET config file -GRID_STAT_CONFIG_FILE = {CONFIG_DIR}/GridStatConfig_REFL +GRID_STAT_CONFIG_FILE = {CONFIG_DIR}/GridStatConfig_REFC GRID_STAT_OUTPUT_PREFIX = {MODEL}_{CURRENT_FCST_NAME}_{OBTYPE} # time looping - options are INIT, VALID, RETRO, and REALTIME diff --git a/ush/valid_param_vals.sh b/ush/valid_param_vals.sh index 96998bc50..d5eb39b9b 100644 --- a/ush/valid_param_vals.sh +++ b/ush/valid_param_vals.sh @@ -52,9 +52,11 @@ valid_vals_RUN_TASK_MAKE_GRID=("TRUE" "true" "YES" "yes" "FALSE" "false" "NO" "n valid_vals_RUN_TASK_MAKE_OROG=("TRUE" "true" "YES" "yes" "FALSE" "false" "NO" "no") valid_vals_RUN_TASK_MAKE_SFC_CLIMO=("TRUE" "true" "YES" "yes" "FALSE" "false" "NO" "no") valid_vals_RUN_TASK_RUN_POST=("TRUE" "true" "YES" "yes" "FALSE" "false" "NO" "no") +valid_vals_WRITE_DOPOST=("TRUE" "true" "YES" "yes" "FALSE" "false" "NO" "no") valid_vals_RUN_TASK_VX_GRIDSTAT=("TRUE" "true" "YES" "yes" "FALSE" "false" "NO" "no") valid_vals_RUN_TASK_VX_POINTSTAT=("TRUE" "true" "YES" "yes" "FALSE" "false" "NO" "no") -valid_vals_WRITE_DOPOST=("TRUE" "true" "YES" "yes" "FALSE" "false" "NO" "no") +valid_vals_RUN_TASK_VX_ENSGRID=("TRUE" "true" "YES" "yes" "FALSE" "false" "NO" "no") +valid_vals_RUN_TASK_VX_ENSPOINT=("TRUE" "true" "YES" "yes" "FALSE" "false" "NO" "no") valid_vals_QUILTING=("TRUE" "true" "YES" "yes" "FALSE" "false" "NO" "no") valid_vals_PRINT_ESMF=("TRUE" "true" "YES" "yes" "FALSE" "false" "NO" "no") valid_vals_USE_CRON_TO_RELAUNCH=("TRUE" "true" "YES" "yes" "FALSE" "false" "NO" "no") From 09ae4f15e3afab0b4eb8a73ecc72306c5a2e8060 Mon Sep 17 00:00:00 2001 From: Michael Kavulich Date: Tue, 21 Sep 2021 21:32:39 -0600 Subject: [PATCH 02/16] Update develop branch for generic linux, MacOS capability (#539) ## DESCRIPTION OF CHANGES: This change will add the capability to run regional_workflow (as part of the SRW app) on MacOS and generic LINUX platforms. Most of these changes are identical to those in #402 (hash bc08607) but some additional modifications needed to be made due to intervening changes in the develop branch. ## TESTS CONDUCTED: Ran Graduate Student Test on new platforms: - my personal Mac machine (MacOS Catalina 10.15.7) MacOS with gnu 9.4.0 compilers. - Cheyenne compute node as a faux "stand-alone" machine, intel 19.1.1 compilers Ran suite of end-to-end tests on Cheyenne (intel/19.1.1) and Hera (intel/18.0.5.274). All passed as expected. Tests also passed on WCOSS, MacOS Mojave, RedHat Linux. ## ISSUE: Will resolve #369 --- jobs/JREGIONAL_GET_EXTRN_MDL_FILES | 2 +- jobs/JREGIONAL_GET_OBS_CCPA | 2 +- jobs/JREGIONAL_GET_OBS_MRMS | 2 +- jobs/JREGIONAL_GET_OBS_NDAS | 2 +- jobs/JREGIONAL_MAKE_GRID | 2 +- jobs/JREGIONAL_MAKE_ICS | 2 +- jobs/JREGIONAL_MAKE_LBCS | 2 +- jobs/JREGIONAL_MAKE_OROG | 2 +- jobs/JREGIONAL_MAKE_SFC_CLIMO | 2 +- jobs/JREGIONAL_RUN_FCST | 2 +- jobs/JREGIONAL_RUN_POST | 4 +- jobs/JREGIONAL_RUN_VX_ENSGRID | 2 +- jobs/JREGIONAL_RUN_VX_ENSGRID_MEAN | 2 +- jobs/JREGIONAL_RUN_VX_ENSGRID_PROB | 2 +- jobs/JREGIONAL_RUN_VX_ENSPOINT | 2 +- jobs/JREGIONAL_RUN_VX_ENSPOINT_MEAN | 2 +- jobs/JREGIONAL_RUN_VX_ENSPOINT_PROB | 2 +- jobs/JREGIONAL_RUN_VX_GRIDSTAT | 2 +- jobs/JREGIONAL_RUN_VX_POINTSTAT | 2 +- scripts/exregional_get_ccpa_files.sh | 8 +- scripts/exregional_get_extrn_mdl_files.sh | 6 +- scripts/exregional_get_mrms_files.sh | 6 +- scripts/exregional_get_ndas_files.sh | 14 +- scripts/exregional_make_grid.sh | 21 +- scripts/exregional_make_ics.sh | 16 +- scripts/exregional_make_lbcs.sh | 18 +- scripts/exregional_make_orog.sh | 36 +-- scripts/exregional_make_sfc_climo.sh | 13 +- scripts/exregional_run_ensgridvx.sh | 2 +- scripts/exregional_run_ensgridvx_mean.sh | 2 +- scripts/exregional_run_ensgridvx_prob.sh | 2 +- scripts/exregional_run_enspointvx.sh | 2 +- scripts/exregional_run_enspointvx_mean.sh | 2 +- scripts/exregional_run_enspointvx_prob.sh | 2 +- scripts/exregional_run_fcst.sh | 23 +- scripts/exregional_run_gridstatvx.sh | 4 +- scripts/exregional_run_pointstatvx.sh | 4 +- scripts/exregional_run_post.sh | 21 +- ush/bash_utils/change_case.sh | 239 ++++++++++++++++++ ush/bash_utils/check_for_preexist_dir_file.sh | 2 +- ush/bash_utils/check_var_valid_value.sh | 2 +- ush/bash_utils/count_files.sh | 2 +- ush/bash_utils/create_symlink_to_file.sh | 2 +- ush/bash_utils/define_macos_utilities.sh | 44 ++++ ush/bash_utils/filesys_cmds_vrfy.sh | 6 +- ush/bash_utils/get_charvar_from_netcdf.sh | 4 +- ush/bash_utils/get_elem_inds.sh | 4 +- .../get_manage_externals_config_property.sh | 6 +- ush/bash_utils/interpol_to_arbit_CRES.sh | 2 +- ush/bash_utils/is_array.sh | 2 +- ush/bash_utils/is_element_of.sh | 2 +- ush/bash_utils/print_input_args.sh | 6 +- ush/bash_utils/print_msg.sh | 8 +- ush/bash_utils/process_args.sh | 4 +- ush/bash_utils/save_restore_shell_opts.sh | 2 +- ush/bash_utils/set_bash_param.sh | 4 +- ush/bash_utils/set_file_param.sh | 4 +- ush/check_ruc_lsm.sh | 4 +- ush/cmp_expt_to_baseline.sh | 2 +- ush/compare_config_scripts.sh | 8 +- ush/config_defaults.sh | 30 +++ ush/create_diag_table_file.sh | 2 +- ush/create_model_configure_file.sh | 10 +- ush/generate_FV3LAM_wflow.sh | 86 ++++--- ush/get_extrn_mdl_file_dir_info.sh | 69 +++-- ush/launch_FV3LAM_wflow.sh | 16 +- ush/link_fix.sh | 4 +- ush/load_modules_run_task.sh | 87 +------ ush/make_grid_mosaic_file.sh | 2 +- ush/set_FV3nml_sfc_climo_filenames.sh | 6 +- ush/set_FV3nml_stoch_params.sh | 2 +- ush/set_cycle_dates.sh | 4 +- ush/set_extrn_mdl_params.sh | 2 +- ush/set_gridparams_ESGgrid.sh | 2 +- ush/set_gridparams_GFDLgrid.sh | 6 +- ush/set_ozone_param.sh | 8 +- ush/set_predef_grid_params.sh | 2 +- ush/set_thompson_mp_fix_files.sh | 4 +- ush/setup.sh | 124 +++++---- ush/source_util_funcs.sh | 24 +- ush/valid_param_vals.sh | 3 +- 81 files changed, 730 insertions(+), 364 deletions(-) create mode 100644 ush/bash_utils/change_case.sh create mode 100644 ush/bash_utils/define_macos_utilities.sh diff --git a/jobs/JREGIONAL_GET_EXTRN_MDL_FILES b/jobs/JREGIONAL_GET_EXTRN_MDL_FILES index 6b373895e..bfe2d7bce 100755 --- a/jobs/JREGIONAL_GET_EXTRN_MDL_FILES +++ b/jobs/JREGIONAL_GET_EXTRN_MDL_FILES @@ -57,7 +57,7 @@ # #----------------------------------------------------------------------- # -scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) scrfunc_fn=$( basename "${scrfunc_fp}" ) scrfunc_dir=$( dirname "${scrfunc_fp}" ) # diff --git a/jobs/JREGIONAL_GET_OBS_CCPA b/jobs/JREGIONAL_GET_OBS_CCPA index 6f541ccd2..e63010d0d 100755 --- a/jobs/JREGIONAL_GET_OBS_CCPA +++ b/jobs/JREGIONAL_GET_OBS_CCPA @@ -39,7 +39,7 @@ # #----------------------------------------------------------------------- # -scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) scrfunc_fn=$( basename "${scrfunc_fp}" ) scrfunc_dir=$( dirname "${scrfunc_fp}" ) # diff --git a/jobs/JREGIONAL_GET_OBS_MRMS b/jobs/JREGIONAL_GET_OBS_MRMS index 4ae197fee..dfe1f68a6 100755 --- a/jobs/JREGIONAL_GET_OBS_MRMS +++ b/jobs/JREGIONAL_GET_OBS_MRMS @@ -35,7 +35,7 @@ # #----------------------------------------------------------------------- # -scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) scrfunc_fn=$( basename "${scrfunc_fp}" ) scrfunc_dir=$( dirname "${scrfunc_fp}" ) # diff --git a/jobs/JREGIONAL_GET_OBS_NDAS b/jobs/JREGIONAL_GET_OBS_NDAS index 32c1d819e..9514931cc 100755 --- a/jobs/JREGIONAL_GET_OBS_NDAS +++ b/jobs/JREGIONAL_GET_OBS_NDAS @@ -35,7 +35,7 @@ # #----------------------------------------------------------------------- # -scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) scrfunc_fn=$( basename "${scrfunc_fp}" ) scrfunc_dir=$( dirname "${scrfunc_fp}" ) # diff --git a/jobs/JREGIONAL_MAKE_GRID b/jobs/JREGIONAL_MAKE_GRID index 641196137..bbf22e024 100755 --- a/jobs/JREGIONAL_MAKE_GRID +++ b/jobs/JREGIONAL_MAKE_GRID @@ -125,7 +125,7 @@ # #----------------------------------------------------------------------- # -scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) scrfunc_fn=$( basename "${scrfunc_fp}" ) scrfunc_dir=$( dirname "${scrfunc_fp}" ) # diff --git a/jobs/JREGIONAL_MAKE_ICS b/jobs/JREGIONAL_MAKE_ICS index 01027b4ac..9c5125f41 100755 --- a/jobs/JREGIONAL_MAKE_ICS +++ b/jobs/JREGIONAL_MAKE_ICS @@ -27,7 +27,7 @@ # #----------------------------------------------------------------------- # -scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) scrfunc_fn=$( basename "${scrfunc_fp}" ) scrfunc_dir=$( dirname "${scrfunc_fp}" ) # diff --git a/jobs/JREGIONAL_MAKE_LBCS b/jobs/JREGIONAL_MAKE_LBCS index b46d0e950..bc7afbaf7 100755 --- a/jobs/JREGIONAL_MAKE_LBCS +++ b/jobs/JREGIONAL_MAKE_LBCS @@ -27,7 +27,7 @@ # #----------------------------------------------------------------------- # -scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) scrfunc_fn=$( basename "${scrfunc_fp}" ) scrfunc_dir=$( dirname "${scrfunc_fp}" ) # diff --git a/jobs/JREGIONAL_MAKE_OROG b/jobs/JREGIONAL_MAKE_OROG index e3935037d..50b6c0369 100755 --- a/jobs/JREGIONAL_MAKE_OROG +++ b/jobs/JREGIONAL_MAKE_OROG @@ -27,7 +27,7 @@ # #----------------------------------------------------------------------- # -scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) scrfunc_fn=$( basename "${scrfunc_fp}" ) scrfunc_dir=$( dirname "${scrfunc_fp}" ) # diff --git a/jobs/JREGIONAL_MAKE_SFC_CLIMO b/jobs/JREGIONAL_MAKE_SFC_CLIMO index f2993bbaf..24af27bfa 100755 --- a/jobs/JREGIONAL_MAKE_SFC_CLIMO +++ b/jobs/JREGIONAL_MAKE_SFC_CLIMO @@ -27,7 +27,7 @@ # #----------------------------------------------------------------------- # -scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) scrfunc_fn=$( basename "${scrfunc_fp}" ) scrfunc_dir=$( dirname "${scrfunc_fp}" ) # diff --git a/jobs/JREGIONAL_RUN_FCST b/jobs/JREGIONAL_RUN_FCST index a4806c765..dbd4c80c4 100755 --- a/jobs/JREGIONAL_RUN_FCST +++ b/jobs/JREGIONAL_RUN_FCST @@ -37,7 +37,7 @@ # #----------------------------------------------------------------------- # -scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) scrfunc_fn=$( basename "${scrfunc_fp}" ) scrfunc_dir=$( dirname "${scrfunc_fp}" ) # diff --git a/jobs/JREGIONAL_RUN_POST b/jobs/JREGIONAL_RUN_POST index 68dcf95bd..ed469dbe2 100755 --- a/jobs/JREGIONAL_RUN_POST +++ b/jobs/JREGIONAL_RUN_POST @@ -36,7 +36,7 @@ # #----------------------------------------------------------------------- # -scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) scrfunc_fn=$( basename "${scrfunc_fp}" ) scrfunc_dir=$( dirname "${scrfunc_fp}" ) # @@ -113,7 +113,7 @@ cd_vrfy "${tmp_dir}" # #----------------------------------------------------------------------- # -fhr=$( printf "%s" "${fhr}" | sed -n -r -e "s/^([0-9]+)$/\1/p" ) +fhr=$( printf "%s" "${fhr}" | $SED -n -r -e "s/^([0-9]+)$/\1/p" ) if [ -z "$fhr" ]; then print_err_msg_exit "\ The forecast hour (fhr) must be a non-empty string consisting of only diff --git a/jobs/JREGIONAL_RUN_VX_ENSGRID b/jobs/JREGIONAL_RUN_VX_ENSGRID index 9fa88c2f7..641df6f66 100755 --- a/jobs/JREGIONAL_RUN_VX_ENSGRID +++ b/jobs/JREGIONAL_RUN_VX_ENSGRID @@ -36,7 +36,7 @@ # #----------------------------------------------------------------------- # -scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) scrfunc_fn=$( basename "${scrfunc_fp}" ) scrfunc_dir=$( dirname "${scrfunc_fp}" ) # diff --git a/jobs/JREGIONAL_RUN_VX_ENSGRID_MEAN b/jobs/JREGIONAL_RUN_VX_ENSGRID_MEAN index 67945ca7b..d8c4dcea0 100755 --- a/jobs/JREGIONAL_RUN_VX_ENSGRID_MEAN +++ b/jobs/JREGIONAL_RUN_VX_ENSGRID_MEAN @@ -36,7 +36,7 @@ # #----------------------------------------------------------------------- # -scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) scrfunc_fn=$( basename "${scrfunc_fp}" ) scrfunc_dir=$( dirname "${scrfunc_fp}" ) # diff --git a/jobs/JREGIONAL_RUN_VX_ENSGRID_PROB b/jobs/JREGIONAL_RUN_VX_ENSGRID_PROB index f792d4b77..7ae543314 100755 --- a/jobs/JREGIONAL_RUN_VX_ENSGRID_PROB +++ b/jobs/JREGIONAL_RUN_VX_ENSGRID_PROB @@ -36,7 +36,7 @@ # #----------------------------------------------------------------------- # -scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) scrfunc_fn=$( basename "${scrfunc_fp}" ) scrfunc_dir=$( dirname "${scrfunc_fp}" ) # diff --git a/jobs/JREGIONAL_RUN_VX_ENSPOINT b/jobs/JREGIONAL_RUN_VX_ENSPOINT index dac8873a2..c240d7abe 100755 --- a/jobs/JREGIONAL_RUN_VX_ENSPOINT +++ b/jobs/JREGIONAL_RUN_VX_ENSPOINT @@ -34,7 +34,7 @@ # #----------------------------------------------------------------------- # -scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) scrfunc_fn=$( basename "${scrfunc_fp}" ) scrfunc_dir=$( dirname "${scrfunc_fp}" ) # diff --git a/jobs/JREGIONAL_RUN_VX_ENSPOINT_MEAN b/jobs/JREGIONAL_RUN_VX_ENSPOINT_MEAN index 3ae1cb56d..376d07b99 100755 --- a/jobs/JREGIONAL_RUN_VX_ENSPOINT_MEAN +++ b/jobs/JREGIONAL_RUN_VX_ENSPOINT_MEAN @@ -34,7 +34,7 @@ # #----------------------------------------------------------------------- # -scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) scrfunc_fn=$( basename "${scrfunc_fp}" ) scrfunc_dir=$( dirname "${scrfunc_fp}" ) # diff --git a/jobs/JREGIONAL_RUN_VX_ENSPOINT_PROB b/jobs/JREGIONAL_RUN_VX_ENSPOINT_PROB index 8810d0bcb..1a47cfd12 100755 --- a/jobs/JREGIONAL_RUN_VX_ENSPOINT_PROB +++ b/jobs/JREGIONAL_RUN_VX_ENSPOINT_PROB @@ -34,7 +34,7 @@ # #----------------------------------------------------------------------- # -scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) scrfunc_fn=$( basename "${scrfunc_fp}" ) scrfunc_dir=$( dirname "${scrfunc_fp}" ) # diff --git a/jobs/JREGIONAL_RUN_VX_GRIDSTAT b/jobs/JREGIONAL_RUN_VX_GRIDSTAT index 300c79f9c..b17fd2960 100755 --- a/jobs/JREGIONAL_RUN_VX_GRIDSTAT +++ b/jobs/JREGIONAL_RUN_VX_GRIDSTAT @@ -36,7 +36,7 @@ # #----------------------------------------------------------------------- # -scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) scrfunc_fn=$( basename "${scrfunc_fp}" ) scrfunc_dir=$( dirname "${scrfunc_fp}" ) # diff --git a/jobs/JREGIONAL_RUN_VX_POINTSTAT b/jobs/JREGIONAL_RUN_VX_POINTSTAT index f67cb5547..f995045ae 100755 --- a/jobs/JREGIONAL_RUN_VX_POINTSTAT +++ b/jobs/JREGIONAL_RUN_VX_POINTSTAT @@ -34,7 +34,7 @@ # #----------------------------------------------------------------------- # -scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) scrfunc_fn=$( basename "${scrfunc_fp}" ) scrfunc_dir=$( dirname "${scrfunc_fp}" ) # diff --git a/scripts/exregional_get_ccpa_files.sh b/scripts/exregional_get_ccpa_files.sh index 3c1504292..b0ab11f5b 100755 --- a/scripts/exregional_get_ccpa_files.sh +++ b/scripts/exregional_get_ccpa_files.sh @@ -47,9 +47,9 @@ while [[ ${current_fcst} -le ${fcst_length} ]]; do mm=`echo ${init} | cut -c5-6` # month (MM) of initialization time dd=`echo ${init} | cut -c7-8` # day (DD) of initialization time hh=`echo ${init} | cut -c9-10` # hour (HH) of initialization time - init_ut=`date -ud ''${yyyy}-${mm}-${dd}' UTC '${hh}':00:00' +%s` # convert initialization time to universal time + init_ut=`$DATE_UTIL -ud ''${yyyy}-${mm}-${dd}' UTC '${hh}':00:00' +%s` # convert initialization time to universal time vdate_ut=`expr ${init_ut} + ${fcst_sec}` # calculate current forecast time in universal time - vdate=`date -ud '1970-01-01 UTC '${vdate_ut}' seconds' +%Y%m%d%H` # convert universal time to standard time + vdate=`$DATE_UTIL -ud '1970-01-01 UTC '${vdate_ut}' seconds' +%Y%m%d%H` # convert universal time to standard time vyyyymmdd=`echo ${vdate} | cut -c1-8` # forecast time (YYYYMMDD) vyyyy=`echo ${vdate} | cut -c1-4` # year (YYYY) of valid time vmm=`echo ${vdate} | cut -c5-6` # month (MM) of valid time @@ -60,7 +60,7 @@ while [[ ${current_fcst} -le ${fcst_length} ]]; do # Calculate valid date - 1 day vdate_ut_m1=`expr ${vdate_ut} - 86400` - vdate_m1=`date -ud '1970-01-01 UTC '${vdate_ut_m1}' seconds' +%Y%m%d%H` + vdate_m1=`$DATE_UTIL -ud '1970-01-01 UTC '${vdate_ut_m1}' seconds' +%Y%m%d%H` vyyyymmdd_m1=`echo ${vdate_m1} | cut -c1-8` vyyyy_m1=`echo ${vdate_m1} | cut -c1-4` vmm_m1=`echo ${vdate_m1} | cut -c5-6` @@ -69,7 +69,7 @@ while [[ ${current_fcst} -le ${fcst_length} ]]; do # Calculate valid date + 1 day vdate_ut_p1=`expr ${vdate_ut} + 86400` - vdate_p1=`date -ud '1970-01-01 UTC '${vdate_ut_p1}' seconds' +%Y%m%d%H` + vdate_p1=`$DATE_UTIL -ud '1970-01-01 UTC '${vdate_ut_p1}' seconds' +%Y%m%d%H` vyyyymmdd_p1=`echo ${vdate_p1} | cut -c1-8` vyyyy_p1=`echo ${vdate_p1} | cut -c1-4` vmm_p1=`echo ${vdate_p1} | cut -c5-6` diff --git a/scripts/exregional_get_extrn_mdl_files.sh b/scripts/exregional_get_extrn_mdl_files.sh index 35e8d373b..ac5127eb8 100755 --- a/scripts/exregional_get_extrn_mdl_files.sh +++ b/scripts/exregional_get_extrn_mdl_files.sh @@ -27,7 +27,7 @@ # #----------------------------------------------------------------------- # -scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) scrfunc_fn=$( basename "${scrfunc_fp}" ) scrfunc_dir=$( dirname "${scrfunc_fp}" ) # @@ -524,7 +524,7 @@ not happen." # to the current directory. Then move the files. # rel_dir=$( printf "%s" "${extrn_mdl_arcvrel_dir}" | \ - sed -r 's%^(\/|\.\/)([^/]*)(.*)%\2\3%' ) + $SED -r 's%^(\/|\.\/)([^/]*)(.*)%\2\3%' ) mv_vrfy ${rel_dir}/* . # # Get the first subdirectory in rel_dir, i.e. the subdirectory before the @@ -533,7 +533,7 @@ not happen." # it. # subdir_to_remove=$( printf "%s" "${rel_dir}" | \ - sed -r 's%^([^/]*)(.*)%\1%' ) + $SED -r 's%^([^/]*)(.*)%\1%' ) rm_vrfy -rf ./${subdir_to_remove} # # If extrn_mdl_arcvrel_dir does not start with a "/" (and it is not diff --git a/scripts/exregional_get_mrms_files.sh b/scripts/exregional_get_mrms_files.sh index 8819959ec..d0abf8f9b 100755 --- a/scripts/exregional_get_mrms_files.sh +++ b/scripts/exregional_get_mrms_files.sh @@ -38,7 +38,7 @@ s_yyyy=`echo ${start_valid} | cut -c1-4` # year (YYYY) of start time s_mm=`echo ${start_valid} | cut -c5-6` # month (MM) of start time s_dd=`echo ${start_valid} | cut -c7-8` # day (DD) of start time s_hh=`echo ${start_valid} | cut -c9-10` # hour (HH) of start time -start_valid_ut=`date -ud ''${s_yyyy}-${s_mm}-${s_dd}' UTC '${s_hh}':00:00' +%s` # convert start time to universal time +start_valid_ut=`$DATE_UTIL -ud ''${s_yyyy}-${s_mm}-${s_dd}' UTC '${s_hh}':00:00' +%s` # convert start time to universal time end_fcst_sec=`expr ${fcst_length} \* 3600` # convert last forecast lead hour to seconds end_valid_ut=`expr ${start_valid_ut} + ${end_fcst_sec}` # calculate current forecast time in universal time @@ -48,7 +48,7 @@ current_fcst=0 fcst_sec=`expr ${current_fcst} \* 3600` # convert forecast lead hour to seconds while [[ ${cur_ut} -le ${end_valid_ut} ]]; do - cur_time=`date -ud '1970-01-01 UTC '${cur_ut}' seconds' +%Y%m%d%H` # convert universal time to standard time + cur_time=`$DATE_UTIL -ud '1970-01-01 UTC '${cur_ut}' seconds' +%Y%m%d%H` # convert universal time to standard time echo "cur_time=${cur_time}" # Calculate valid date info @@ -57,7 +57,7 @@ while [[ ${cur_ut} -le ${end_valid_ut} ]]; do vdd=`echo ${cur_time} | cut -c7-8` # day (DD) of time vhh=`echo ${cur_time} | cut -c9-10` # hour (HH) of time vyyyymmdd=`echo ${cur_time} | cut -c1-8` # YYYYMMDD of time - vinit_ut=`date -ud ''${vyyyy}-${vmm}-${vdd}' UTC '${vhh}':00:00' +%s` # convert time to universal time + vinit_ut=`$DATE_UTIL -ud ''${vyyyy}-${vmm}-${vdd}' UTC '${vhh}':00:00' +%s` # convert time to universal time # Create necessary raw and proc directories if [[ ! -d "$mrms_raw/${vyyyymmdd}" ]]; then diff --git a/scripts/exregional_get_ndas_files.sh b/scripts/exregional_get_ndas_files.sh index e1ec7f8d6..0105f2edb 100755 --- a/scripts/exregional_get_ndas_files.sh +++ b/scripts/exregional_get_ndas_files.sh @@ -40,9 +40,9 @@ while [[ ${current_fcst} -le ${fcst_length} ]]; do mm=`echo ${init} | cut -c5-6` # month (MM) of initialization time dd=`echo ${init} | cut -c7-8` # day (DD) of initialization time hh=`echo ${init} | cut -c9-10` # hour (HH) of initialization time - init_ut=`date -ud ''${yyyy}-${mm}-${dd}' UTC '${hh}':00:00' +%s` # convert initialization time to universal time + init_ut=`$DATE_UTIL -ud ''${yyyy}-${mm}-${dd}' UTC '${hh}':00:00' +%s` # convert initialization time to universal time vdate_ut=`expr ${init_ut} + ${fcst_sec}` # calculate current forecast time in universal time - vdate=`date -ud '1970-01-01 UTC '${vdate_ut}' seconds' +%Y%m%d%H` # convert universal time to standard time + vdate=`$DATE_UTIL -ud '1970-01-01 UTC '${vdate_ut}' seconds' +%Y%m%d%H` # convert universal time to standard time vyyyymmdd=`echo ${vdate} | cut -c1-8` # forecast time (YYYYMMDD) vyyyy=`echo ${vdate} | cut -c1-4` # year (YYYY) of valid time vmm=`echo ${vdate} | cut -c5-6` # month (MM) of valid time @@ -53,7 +53,7 @@ echo "yyyy mm dd hh= $yyyy $mm $dd $hh" echo "vyyyy vmm vdd vhh= $vyyyy $vmm $vdd $vhh" vdate_ut_m1h=`expr ${vdate_ut} - 3600` # calculate current forecast time in universal time - vdate_m1h=`date -ud '1970-01-01 UTC '${vdate_ut_m1h}' seconds' +%Y%m%d%H` # convert universal time to standard time + vdate_m1h=`$DATE_UTIL -ud '1970-01-01 UTC '${vdate_ut_m1h}' seconds' +%Y%m%d%H` # convert universal time to standard time vyyyymmdd_m1h=`echo ${vdate_m1h} | cut -c1-8` # forecast time (YYYYMMDD) vyyyy_m1h=`echo ${vdate_m1h} | cut -c1-4` # year (YYYY) of valid time vmm_m1h=`echo ${vdate_m1h} | cut -c5-6` # month (MM) of valid time @@ -61,7 +61,7 @@ echo "vyyyy vmm vdd vhh= $vyyyy $vmm $vdd $vhh" vhh_m1h=`echo ${vdate_m1h} | cut -c9-10` # forecast hour (HH) vdate_ut_m2h=`expr ${vdate_ut} - 7200` # calculate current forecast time in universal time - vdate_m2h=`date -ud '1970-01-01 UTC '${vdate_ut_m2h}' seconds' +%Y%m%d%H` # convert universal time to standard time + vdate_m2h=`$DATE_UTIL -ud '1970-01-01 UTC '${vdate_ut_m2h}' seconds' +%Y%m%d%H` # convert universal time to standard time vyyyymmdd_m2h=`echo ${vdate_m2h} | cut -c1-8` # forecast time (YYYYMMDD) vyyyy_m2h=`echo ${vdate_m2h} | cut -c1-4` # year (YYYY) of valid time vmm_m2h=`echo ${vdate_m2h} | cut -c5-6` # month (MM) of valid time @@ -69,7 +69,7 @@ echo "vyyyy vmm vdd vhh= $vyyyy $vmm $vdd $vhh" vhh_m2h=`echo ${vdate_m2h} | cut -c9-10` # forecast hour (HH) vdate_ut_m3h=`expr ${vdate_ut} - 10800` # calculate current forecast time in universal time - vdate_m3h=`date -ud '1970-01-01 UTC '${vdate_ut_m3h}' seconds' +%Y%m%d%H` # convert universal time to standard time + vdate_m3h=`$DATE_UTIL -ud '1970-01-01 UTC '${vdate_ut_m3h}' seconds' +%Y%m%d%H` # convert universal time to standard time vyyyymmdd_m3h=`echo ${vdate_m3h} | cut -c1-8` # forecast time (YYYYMMDD) vyyyy_m3h=`echo ${vdate_m3h} | cut -c1-4` # year (YYYY) of valid time vmm_m3h=`echo ${vdate_m3h} | cut -c5-6` # month (MM) of valid time @@ -77,7 +77,7 @@ echo "vyyyy vmm vdd vhh= $vyyyy $vmm $vdd $vhh" vhh_m3h=`echo ${vdate_m3h} | cut -c9-10` # forecast hour (HH) vdate_ut_m4h=`expr ${vdate_ut} - 14400` # calculate current forecast time in universal time - vdate_m4h=`date -ud '1970-01-01 UTC '${vdate_ut_m4h}' seconds' +%Y%m%d%H` # convert universal time to standard time + vdate_m4h=`$DATE_UTIL -ud '1970-01-01 UTC '${vdate_ut_m4h}' seconds' +%Y%m%d%H` # convert universal time to standard time vyyyymmdd_m4h=`echo ${vdate_m4h} | cut -c1-8` # forecast time (YYYYMMDD) vyyyy_m4h=`echo ${vdate_m4h} | cut -c1-4` # year (YYYY) of valid time vmm_m4h=`echo ${vdate_m4h} | cut -c5-6` # month (MM) of valid time @@ -85,7 +85,7 @@ echo "vyyyy vmm vdd vhh= $vyyyy $vmm $vdd $vhh" vhh_m4h=`echo ${vdate_m4h} | cut -c9-10` # forecast hour (HH) vdate_ut_m5h=`expr ${vdate_ut} - 18000` # calculate current forecast time in universal time - vdate_m5h=`date -ud '1970-01-01 UTC '${vdate_ut_m5h}' seconds' +%Y%m%d%H` # convert universal time to standard time + vdate_m5h=`$DATE_UTIL -ud '1970-01-01 UTC '${vdate_ut_m5h}' seconds' +%Y%m%d%H` # convert universal time to standard time vyyyymmdd_m5h=`echo ${vdate_m5h} | cut -c1-8` # forecast time (YYYYMMDD) vyyyy_m5h=`echo ${vdate_m5h} | cut -c1-4` # year (YYYY) of valid time vmm_m5h=`echo ${vdate_m5h} | cut -c5-6` # month (MM) of valid time diff --git a/scripts/exregional_make_grid.sh b/scripts/exregional_make_grid.sh index 5d9bddaf6..9d4a1d9f6 100755 --- a/scripts/exregional_make_grid.sh +++ b/scripts/exregional_make_grid.sh @@ -37,7 +37,7 @@ # #----------------------------------------------------------------------- # -scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) scrfunc_fn=$( basename "${scrfunc_fp}" ) scrfunc_dir=$( dirname "${scrfunc_fp}" ) # @@ -135,11 +135,18 @@ case "$MACHINE" in ulimit -a ;; + "MACOS") + APRUN=time + ;; + + "LINUX") + APRUN=time + ;; + *) print_err_msg_exit "\ Run command has not been specified for this machine: - MACHINE = \"$MACHINE\" - APRUN = \"$APRUN\"" + MACHINE = \"$MACHINE\"" ;; esac @@ -422,6 +429,14 @@ uniform cubed-sphere grid equivalent resolution returned with nonzero exit code: exec_fp = \"${exec_fp}\"" +# Make sure 'ncdump' is available before we try to use it +if ! command -v ncdump &> /dev/null +then + print_err_msg_exit "\ +The utility 'ncdump' was not found in the environment. Be sure to add the +netCDF 'bin/' directory to your PATH." +fi + # Make the following (reading of res_equiv) a function in another file # so that it can be used both here and in the exregional_make_orog.sh # script. diff --git a/scripts/exregional_make_ics.sh b/scripts/exregional_make_ics.sh index 41b00ac63..ec0fee623 100755 --- a/scripts/exregional_make_ics.sh +++ b/scripts/exregional_make_ics.sh @@ -27,7 +27,7 @@ # #----------------------------------------------------------------------- # -scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) scrfunc_fn=$( basename "${scrfunc_fp}" ) scrfunc_dir=$( dirname "${scrfunc_fp}" ) # @@ -126,6 +126,20 @@ case "$MACHINE" in APRUN="ibrun" ;; + "MACOS") + APRUN=$RUN_CMD_UTILS + ;; + + "LINUX") + APRUN=$RUN_CMD_UTILS + ;; + + *) + print_err_msg_exit "\ +Run command has not been specified for this machine: + MACHINE = \"$MACHINE\"" + ;; + esac # #----------------------------------------------------------------------- diff --git a/scripts/exregional_make_lbcs.sh b/scripts/exregional_make_lbcs.sh index 3865f092a..1b1f84ff8 100755 --- a/scripts/exregional_make_lbcs.sh +++ b/scripts/exregional_make_lbcs.sh @@ -27,7 +27,7 @@ # #----------------------------------------------------------------------- # -scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) scrfunc_fn=$( basename "${scrfunc_fp}" ) scrfunc_dir=$( dirname "${scrfunc_fp}" ) # @@ -126,6 +126,20 @@ case "$MACHINE" in APRUN="ibrun" ;; + "MACOS") + APRUN=$RUN_CMD_UTILS + ;; + + "LINUX") + APRUN=$RUN_CMD_UTILS + ;; + + *) + print_err_msg_exit "\ +Run command has not been specified for this machine: + MACHINE = \"$MACHINE\"" + ;; + esac # #----------------------------------------------------------------------- @@ -438,7 +452,7 @@ list file has not specified for this external LBC model (EXTRN_MDL_NAME_LBCS): dd="${EXTRN_MDL_CDATE:6:2}" hh="${EXTRN_MDL_CDATE:8:2}" - cdate_crnt_fhr=$( date --utc --date "${yyyymmdd} ${hh} UTC + ${fhr} hours" "+%Y%m%d%H" ) + cdate_crnt_fhr=$( $DATE_UTIL --utc --date "${yyyymmdd} ${hh} UTC + ${fhr} hours" "+%Y%m%d%H" ) # # Get the month, day, and hour corresponding to the current forecast time # of the the external model. diff --git a/scripts/exregional_make_orog.sh b/scripts/exregional_make_orog.sh index ed7d43a0c..e90b3b762 100755 --- a/scripts/exregional_make_orog.sh +++ b/scripts/exregional_make_orog.sh @@ -35,7 +35,7 @@ # #----------------------------------------------------------------------- # -scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) scrfunc_fn=$( basename "${scrfunc_fp}" ) scrfunc_dir=$( dirname "${scrfunc_fp}" ) # @@ -147,11 +147,18 @@ case "$MACHINE" in APRUN="time" ;; + "MACOS") + APRUN=time + ;; + + "LINUX") + APRUN=time + ;; + *) print_err_msg_exit "\ Run command has not been specified for this machine: - MACHINE = \"$MACHINE\" - APRUN = \"$APRUN\"" + MACHINE = \"$MACHINE\"" ;; esac @@ -226,7 +233,8 @@ cp_vrfy ${TOPO_DIR}/gmted2010.30sec.int fort.235 mosaic_fn="${CRES}${DOT_OR_USCORE}mosaic.halo${NHW}.nc" mosaic_fp="$FIXLAM/${mosaic_fn}" -grid_fn=$( get_charvar_from_netcdf "${mosaic_fp}" "gridfiles" ) +grid_fn=$( get_charvar_from_netcdf "${mosaic_fp}" "gridfiles" ) || print_err_msg_exit "\ + get_charvar_from_netcdf function failed." grid_fp="${FIXLAM}/${grid_fn}" # #----------------------------------------------------------------------- @@ -284,7 +292,7 @@ cat "${input_redirect_fn}" # #----------------------------------------------------------------------- # -print_info_msg "$VERBOSE" " +print_info_msg "$VERBOSE" "\ Starting orography file generation..." $APRUN "${exec_fp}" < "${input_redirect_fn}" || \ @@ -327,18 +335,17 @@ if [ "${CCPP_PHYS_SUITE}" = "FV3_HRRR" ]; then cd_vrfy ${tmp_dir} mosaic_fn_gwd="${CRES}${DOT_OR_USCORE}mosaic.halo${NH4}.nc" mosaic_fp_gwd="$FIXLAM/${mosaic_fn_gwd}" - grid_fn_gwd=$( get_charvar_from_netcdf "${mosaic_fp_gwd}" "gridfiles" ) + grid_fn_gwd=$( get_charvar_from_netcdf "${mosaic_fp_gwd}" "gridfiles" ) || \ + print_err_msg_exit "get_charvar_from_netcdf function failed." grid_fp_gwd="${FIXLAM}/${grid_fn_gwd}" ls_fn="geo_em.d01.lat-lon.2.5m.HGT_M.nc" ss_fn="HGT.Beljaars_filtered.lat-lon.30s_res.nc" - if [ "${MACHINE}" = "WCOSS_CRAY" ]; then - relative_or_null="" - else - relative_or_null="--relative" - fi - ln_vrfy -fs ${relative_or_null} "${grid_fp_gwd}" "${tmp_dir}/${grid_fn_gwd}" - ln_vrfy -fs ${relative_or_null} "${FIXam}/${ls_fn}" "${tmp_dir}/${ls_fn}" - ln_vrfy -fs ${relative_or_null} "${FIXam}/${ss_fn}" "${tmp_dir}/${ss_fn}" + create_symlink_to_file target="${grid_fp_gwd}" symlink="${tmp_dir}/${grid_fn_gwd}" \ + relative="TRUE" + create_symlink_to_file target="${FIXam}/${ls_fn}" symlink="${tmp_dir}/${ls_fn}" \ + relative="TRUE" + create_symlink_to_file target="${FIXam}/${ss_fn}" symlink="${tmp_dir}/${ss_fn}" \ + relative="TRUE" input_redirect_fn="grid_info.dat" cat > "${input_redirect_fn}" < /dev/null 2>&1 +# +#----------------------------------------------------------------------- +# +# Get the full path to the file in which this script/function is located +# (scrfunc_fp), the name of that file (scrfunc_fn), and the directory in +# which the file is located (scrfunc_dir). +# +#----------------------------------------------------------------------- +# + local scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) + local scrfunc_fn=$( basename "${scrfunc_fp}" ) + local scrfunc_dir=$( dirname "${scrfunc_fp}" ) +# +#----------------------------------------------------------------------- +# +# Get the name of this function. +# +#----------------------------------------------------------------------- +# + local func_name="${FUNCNAME[0]}" +# +#----------------------------------------------------------------------- +# +# Get information about the script or function that calls this function. +# Note that caller_name will be set as follows: +# +# 1) If the caller is a function, caller_name will be set to the name of +# that function. +# 2) If the caller is a sourced script, caller_name will be set to +# "script". Note that a sourced script cannot be the top level +# script since by defintion, it is sourced by another script or func- +# tion. +# 3) If the caller is the top-level script, caller_name will be set to +# "main". +# +# Thus, if caller_name is set to "script" or "main", the caller is a +# script, and if it is set to anything else, the caller is a function. +# +#----------------------------------------------------------------------- +# + local caller_fp=$( $READLINK -f "${BASH_SOURCE[1]}" ) + local caller_fn=$( basename "${caller_fp}" ) + local caller_dir=$( dirname "${caller_fp}" ) + local caller_name="${FUNCNAME[1]}" +# +# Get input string + + local input + + if [ "$#" -eq 1 ]; then + + input="$1" + +# +#----------------------------------------------------------------------- +# +# If no arguments or more than one, print out a usage message and exit. +# +#----------------------------------------------------------------------- +# + else + + print_err_msg_exit " +Incorrect number of arguments specified: + + Function name: \"${func_name}\" + Number of arguments specified: $# + +Usage: + + ${func_name} string + +where: + + string: + This is the string that should be converted to uppercase and echoed. +" + + fi + +# Echo the input string as upperercase + +echo $input| tr '[a-z]' '[A-Z]' + +# +#----------------------------------------------------------------------- +# +# Restore the shell options saved at the beginning of this script/func- +# tion. +# +#----------------------------------------------------------------------- +# + { restore_shell_opts; } > /dev/null 2>&1 + + +} + + +# +#----------------------------------------------------------------------- +# +# Function to echo the given string as a lowercase string +# +#----------------------------------------------------------------------- +# +function echo_lowercase() { +# +#----------------------------------------------------------------------- +# +# Save current shell options (in a global array). Then set new options +# for this script/function. +# +#----------------------------------------------------------------------- +# + { save_shell_opts; set -u +x; } > /dev/null 2>&1 +# +#----------------------------------------------------------------------- +# +# Get the full path to the file in which this script/function is located +# (scrfunc_fp), the name of that file (scrfunc_fn), and the directory in +# which the file is located (scrfunc_dir). +# +#----------------------------------------------------------------------- +# + local scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) + local scrfunc_fn=$( basename "${scrfunc_fp}" ) + local scrfunc_dir=$( dirname "${scrfunc_fp}" ) +# +#----------------------------------------------------------------------- +# +# Get the name of this function. +# +#----------------------------------------------------------------------- +# + local func_name="${FUNCNAME[0]}" +# +#----------------------------------------------------------------------- +# +# Get information about the script or function that calls this function. +# Note that caller_name will be set as follows: +# +# 1) If the caller is a function, caller_name will be set to the name of +# that function. +# 2) If the caller is a sourced script, caller_name will be set to +# "script". Note that a sourced script cannot be the top level +# script since by defintion, it is sourced by another script or func- +# tion. +# 3) If the caller is the top-level script, caller_name will be set to +# "main". +# +# Thus, if caller_name is set to "script" or "main", the caller is a +# script, and if it is set to anything else, the caller is a function. +# +#----------------------------------------------------------------------- +# + local caller_fp=$( $READLINK -f "${BASH_SOURCE[1]}" ) + local caller_fn=$( basename "${caller_fp}" ) + local caller_dir=$( dirname "${caller_fp}" ) + local caller_name="${FUNCNAME[1]}" +# +# Get input string + + local input + + if [ "$#" -eq 1 ]; then + + input="$1" + +# +#----------------------------------------------------------------------- +# +# If no arguments or more than one, print out a usage message and exit. +# +#----------------------------------------------------------------------- +# + else + + print_err_msg_exit " +Incorrect number of arguments specified: + + Function name: \"${func_name}\" + Number of arguments specified: $# + +Usage: + + ${func_name} string + +where: + + string: + This is the string that should be converted to lowercase and echoed. +" + + fi + +# Echo the input string as lowercase + +echo $input| tr '[A-Z]' '[a-z]' + +# +#----------------------------------------------------------------------- +# +# Restore the shell options saved at the beginning of this script/func- +# tion. +# +#----------------------------------------------------------------------- +# + { restore_shell_opts; } > /dev/null 2>&1 + + +} + diff --git a/ush/bash_utils/check_for_preexist_dir_file.sh b/ush/bash_utils/check_for_preexist_dir_file.sh index 7b9b59772..dac2688a3 100644 --- a/ush/bash_utils/check_for_preexist_dir_file.sh +++ b/ush/bash_utils/check_for_preexist_dir_file.sh @@ -26,7 +26,7 @@ function check_for_preexist_dir_file() { # #----------------------------------------------------------------------- # - local scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) + local scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) local scrfunc_fn=$( basename "${scrfunc_fp}" ) local scrfunc_dir=$( dirname "${scrfunc_fp}" ) # diff --git a/ush/bash_utils/check_var_valid_value.sh b/ush/bash_utils/check_var_valid_value.sh index 723460ea1..7a0e20707 100644 --- a/ush/bash_utils/check_var_valid_value.sh +++ b/ush/bash_utils/check_var_valid_value.sh @@ -25,7 +25,7 @@ function check_var_valid_value() { # #----------------------------------------------------------------------- # - local scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) + local scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) local scrfunc_fn=$( basename "${scrfunc_fp}" ) local scrfunc_dir=$( dirname "${scrfunc_fp}" ) # diff --git a/ush/bash_utils/count_files.sh b/ush/bash_utils/count_files.sh index 633bbf7ce..c80f342cb 100644 --- a/ush/bash_utils/count_files.sh +++ b/ush/bash_utils/count_files.sh @@ -25,7 +25,7 @@ function count_files() { # #----------------------------------------------------------------------- # - local scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) + local scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) local scrfunc_fn=$( basename "${scrfunc_fp}" ) local scrfunc_dir=$( dirname "${scrfunc_fp}" ) # diff --git a/ush/bash_utils/create_symlink_to_file.sh b/ush/bash_utils/create_symlink_to_file.sh index e54bf0a36..c14a35bdc 100644 --- a/ush/bash_utils/create_symlink_to_file.sh +++ b/ush/bash_utils/create_symlink_to_file.sh @@ -31,7 +31,7 @@ function create_symlink_to_file() { # #----------------------------------------------------------------------- # - local scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) + local scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) local scrfunc_fn=$( basename "${scrfunc_fp}" ) local scrfunc_dir=$( dirname "${scrfunc_fp}" ) # diff --git a/ush/bash_utils/define_macos_utilities.sh b/ush/bash_utils/define_macos_utilities.sh new file mode 100644 index 000000000..d89780944 --- /dev/null +++ b/ush/bash_utils/define_macos_utilities.sh @@ -0,0 +1,44 @@ +# +#----------------------------------------------------------------------- +# +# This script defines MacOS-specific UNIX command-line utilities that +# mimic the functionality of the GNU equivalents. +# +#----------------------------------------------------------------------- +# +# +# +#----------------------------------------------------------------------- +# +# Check if we are on a Darwin machine; if so we need to use the gnu-like +# equivalent of readlink and sed. +# +#----------------------------------------------------------------------- +# +darwinerror () { + + utility=$1 + echo >&2 " +For Darwin-based operating systems (MacOS), the '${utility}' utility is required to run the UFS SRW Application. +Reference the User's Guide for more information about platform requirements. +Aborting. +" + exit 1 +} + + if [[ $(uname -s) == Darwin ]]; then + export READLINK=greadlink + command -v $READLINK >/dev/null 2>&1 || darwinerror $READLINK + export SED=gsed + command -v $SED >/dev/null 2>&1 || darwinerror $SED + export DATE_UTIL=gdate + command -v $DATE_UTIL >/dev/null 2>&1 || darwinerror $DATE_UTIL + export LN_UTIL=gln + command -v $LN_UTIL >/dev/null 2>&1 || darwinerror $LN_UTIL + else + export READLINK=readlink + export SED=sed + export DATE_UTIL=date + export LN_UTIL=ln + fi + diff --git a/ush/bash_utils/filesys_cmds_vrfy.sh b/ush/bash_utils/filesys_cmds_vrfy.sh index 57a087058..d147fc107 100644 --- a/ush/bash_utils/filesys_cmds_vrfy.sh +++ b/ush/bash_utils/filesys_cmds_vrfy.sh @@ -28,7 +28,7 @@ function filesys_cmd_vrfy() { # #----------------------------------------------------------------------- # - local scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) + local scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) local scrfunc_fn=$( basename "${scrfunc_fp}" ) local scrfunc_dir=$( dirname "${scrfunc_fp}" ) # @@ -65,7 +65,7 @@ function filesys_cmd_vrfy() { # #----------------------------------------------------------------------- # - local caller_fp=$( readlink -f "${BASH_SOURCE[2]}" ) + local caller_fp=$( $READLINK -f "${BASH_SOURCE[2]}" ) local caller_fn=$( basename "${caller_fp}" ) local caller_dir=$( dirname "${caller_fp}" ) local caller_name="${FUNCNAME[2]}" @@ -258,7 +258,7 @@ function rm_vrfy() { function ln_vrfy() { { save_shell_opts; set -u +x; } > /dev/null 2>&1 - filesys_cmd_vrfy "ln" "$@" + filesys_cmd_vrfy "$LN_UTIL" "$@" { restore_shell_opts; } > /dev/null 2>&1 } diff --git a/ush/bash_utils/get_charvar_from_netcdf.sh b/ush/bash_utils/get_charvar_from_netcdf.sh index 71bcf9dd0..ac70cf68d 100644 --- a/ush/bash_utils/get_charvar_from_netcdf.sh +++ b/ush/bash_utils/get_charvar_from_netcdf.sh @@ -35,7 +35,7 @@ function get_charvar_from_netcdf() { # #----------------------------------------------------------------------- # - local scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) + local scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) local scrfunc_fn=$( basename "${scrfunc_fp}" ) local scrfunc_dir=$( dirname "${scrfunc_fp}" ) # @@ -173,7 +173,7 @@ The arguments to this function are defined as follows: #----------------------------------------------------------------------- # nc_var_value=$( ncdump -v "${nc_var_name}" "${nc_file}" | \ - sed -r -e '1,/data:/d' \ + $SED -r -e '1,/data:/d' \ -e '/^[ ]*'${nc_var_name}'/d' \ -e '/^}$/d' \ -e 's/.*"(.*)".*/\1/' \ diff --git a/ush/bash_utils/get_elem_inds.sh b/ush/bash_utils/get_elem_inds.sh index 70da5b248..89a9c4122 100644 --- a/ush/bash_utils/get_elem_inds.sh +++ b/ush/bash_utils/get_elem_inds.sh @@ -24,7 +24,7 @@ function get_elem_inds() { # #----------------------------------------------------------------------- # - local scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) + local scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) local scrfunc_fn=$( basename "${scrfunc_fp}" ) local scrfunc_dir=$( dirname "${scrfunc_fp}" ) # @@ -123,7 +123,7 @@ The arguments to this function are defined as follows: # #----------------------------------------------------------------------- # - inds_to_return="${inds_to_return,,}" + inds_to_return=$(echo_lowercase $inds_to_return) valid_vals_inds_to_return=( "first" "last" "all" ) check_var_valid_value "inds_to_return" "valid_vals_inds_to_return" # diff --git a/ush/bash_utils/get_manage_externals_config_property.sh b/ush/bash_utils/get_manage_externals_config_property.sh index b42605ec9..370d448f2 100644 --- a/ush/bash_utils/get_manage_externals_config_property.sh +++ b/ush/bash_utils/get_manage_externals_config_property.sh @@ -24,7 +24,7 @@ function get_manage_externals_config_property() { # #----------------------------------------------------------------------- # - local scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) + local scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) local scrfunc_fn=$( basename "${scrfunc_fp}" ) local scrfunc_dir=$( dirname "${scrfunc_fp}" ) # @@ -184,7 +184,7 @@ does not exist: #----------------------------------------------------------------------- # regex_search="^[ ]*(${property_name})[ ]*=[ ]*([^ ]*).*" - line=$( sed -r -n \ + line=$( $SED -r -n \ -e "/^[ ]*\[${external_name}\]/!b" \ -e ":SearchForLine" \ -e "s/(${regex_search})/\1/;t FoundLine" \ @@ -225,7 +225,7 @@ fied external (external_name): else property_value=$( printf "%s" "${line}" | \ - sed -r -n -e "s/${regex_search}/\2/p" ) + $SED -r -n -e "s/${regex_search}/\2/p" ) printf "%s\n" "${property_value}" fi diff --git a/ush/bash_utils/interpol_to_arbit_CRES.sh b/ush/bash_utils/interpol_to_arbit_CRES.sh index 6f685c1a5..bc4c6e833 100644 --- a/ush/bash_utils/interpol_to_arbit_CRES.sh +++ b/ush/bash_utils/interpol_to_arbit_CRES.sh @@ -27,7 +27,7 @@ function interpol_to_arbit_CRES() { # #----------------------------------------------------------------------- # - local scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) + local scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) local scrfunc_fn=$( basename "${scrfunc_fp}" ) local scrfunc_dir=$( dirname "${scrfunc_fp}" ) # diff --git a/ush/bash_utils/is_array.sh b/ush/bash_utils/is_array.sh index 16be2fbb7..c831f313e 100644 --- a/ush/bash_utils/is_array.sh +++ b/ush/bash_utils/is_array.sh @@ -31,7 +31,7 @@ function is_array() { # #----------------------------------------------------------------------- # - local scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) + local scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) local scrfunc_fn=$( basename "${scrfunc_fp}" ) local scrfunc_dir=$( dirname "${scrfunc_fp}" ) # diff --git a/ush/bash_utils/is_element_of.sh b/ush/bash_utils/is_element_of.sh index d3060db7d..3ae9ce1cc 100644 --- a/ush/bash_utils/is_element_of.sh +++ b/ush/bash_utils/is_element_of.sh @@ -24,7 +24,7 @@ function is_element_of() { # #----------------------------------------------------------------------- # - local scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) + local scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) local scrfunc_fn=$( basename "${scrfunc_fp}" ) local scrfunc_dir=$( dirname "${scrfunc_fp}" ) # diff --git a/ush/bash_utils/print_input_args.sh b/ush/bash_utils/print_input_args.sh index 10bbe4ca6..72b7744b3 100644 --- a/ush/bash_utils/print_input_args.sh +++ b/ush/bash_utils/print_input_args.sh @@ -31,7 +31,7 @@ function print_input_args() { # #----------------------------------------------------------------------- # - local scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) + local scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) local scrfunc_fn=$( basename "${scrfunc_fp}" ) local scrfunc_dir=$( dirname "${scrfunc_fp}" ) # @@ -62,7 +62,7 @@ function print_input_args() { # #----------------------------------------------------------------------- # - local caller_fp=$( readlink -f "${BASH_SOURCE[1]}" ) + local caller_fp=$( $READLINK -f "${BASH_SOURCE[1]}" ) local caller_fn=$( basename "${caller_fp}" ) local caller_dir=$( dirname "${caller_fp}" ) local caller_name="${FUNCNAME[1]}" @@ -174,7 +174,7 @@ have been set as follows: # #----------------------------------------------------------------------- # - if [ ! -v VERBOSE ]; then + if [ -z ${VERBOSE+x} ]; then print_info_msg "$msg" else print_info_msg "$VERBOSE" "$msg" diff --git a/ush/bash_utils/print_msg.sh b/ush/bash_utils/print_msg.sh index 544357f62..d83732942 100644 --- a/ush/bash_utils/print_msg.sh +++ b/ush/bash_utils/print_msg.sh @@ -33,7 +33,7 @@ function print_info_msg() { # #----------------------------------------------------------------------- # - local scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) + local scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) local scrfunc_fn=$( basename "${scrfunc_fp}" ) local scrfunc_dir=$( dirname "${scrfunc_fp}" ) # @@ -64,7 +64,7 @@ function print_info_msg() { # #----------------------------------------------------------------------- # - local caller_fp=$( readlink -f "${BASH_SOURCE[1]}" ) + local caller_fp=$( $READLINK -f "${BASH_SOURCE[1]}" ) local caller_fn=$( basename "${caller_fp}" ) local caller_dir=$( dirname "${caller_fp}" ) local caller_name="${FUNCNAME[1]}" @@ -182,7 +182,7 @@ function print_err_msg_exit() { # #----------------------------------------------------------------------- # - local scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) + local scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) local scrfunc_fn=$( basename "${scrfunc_fp}" ) local scrfunc_dir=$( dirname "${scrfunc_fp}" ) # @@ -213,7 +213,7 @@ function print_err_msg_exit() { # #----------------------------------------------------------------------- # - local caller_fp=$( readlink -f "${BASH_SOURCE[1]}" ) + local caller_fp=$( $READLINK -f "${BASH_SOURCE[1]}" ) local caller_fn=$( basename "${caller_fp}" ) local caller_dir=$( dirname "${caller_fp}" ) local caller_name="${FUNCNAME[1]}" diff --git a/ush/bash_utils/process_args.sh b/ush/bash_utils/process_args.sh index 4363348ba..203bdedcf 100644 --- a/ush/bash_utils/process_args.sh +++ b/ush/bash_utils/process_args.sh @@ -99,7 +99,7 @@ function process_args() { # #----------------------------------------------------------------------- # - local scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) + local scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) local scrfunc_fn=$( basename "${scrfunc_fp}" ) local scrfunc_dir=$( dirname "${scrfunc_fp}" ) # @@ -263,7 +263,7 @@ ments (num_valid_args) specified in the array valid_arg_names: # Remove spaces (if any exist) from the current valid argument name. valid_arg_name_no_spaces=$( \ - printf "%s\n" "${valid_arg_name}" | sed -r -e 's/[[:space:]]//g' ) + printf "%s\n" "${valid_arg_name}" | $SED -r -e 's/[[:space:]]//g' ) if [ "${valid_arg_name_no_spaces}" != "${valid_arg_name}" ]; then print_err_msg_exit "\ diff --git a/ush/bash_utils/save_restore_shell_opts.sh b/ush/bash_utils/save_restore_shell_opts.sh index 37f699590..b561b6cea 100644 --- a/ush/bash_utils/save_restore_shell_opts.sh +++ b/ush/bash_utils/save_restore_shell_opts.sh @@ -19,7 +19,7 @@ function save_shell_opts() { # local shell_opts="$(set +o)"$'\n'"set -$-" shell_opts=${shell_opts//$'\n'/ } - shell_opts=$( printf "%s\n" "$shell_opts" | sed -r -e "s/set ([+-])/\1/g" ) + shell_opts=$( printf "%s\n" "$shell_opts" | $SED -r -e "s/set ([+-])/\1/g" ) # # Store the current set of shell options in the global array shell_- # opts_array so we can reuse them later. diff --git a/ush/bash_utils/set_bash_param.sh b/ush/bash_utils/set_bash_param.sh index 12c2e49be..dd2220607 100644 --- a/ush/bash_utils/set_bash_param.sh +++ b/ush/bash_utils/set_bash_param.sh @@ -25,7 +25,7 @@ function set_bash_param() { # #----------------------------------------------------------------------- # - local scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) + local scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) local scrfunc_fn=$( basename "${scrfunc_fp}" ) local scrfunc_dir=$( dirname "${scrfunc_fp}" ) # @@ -129,7 +129,7 @@ lar expression (regex_search): regex_search = ${regex_search}" }; - sed -i -r -e "s%${regex_search}%${regex_replace}%" "${file_full_path}" + $SED -i -r -e "s%${regex_search}%${regex_replace}%" "${file_full_path}" # #----------------------------------------------------------------------- # diff --git a/ush/bash_utils/set_file_param.sh b/ush/bash_utils/set_file_param.sh index 72a9ab041..d45451483 100644 --- a/ush/bash_utils/set_file_param.sh +++ b/ush/bash_utils/set_file_param.sh @@ -25,7 +25,7 @@ function set_file_param() { # #----------------------------------------------------------------------- # - local scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) + local scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) local scrfunc_fn=$( basename "${scrfunc_fp}" ) local scrfunc_dir=$( dirname "${scrfunc_fp}" ) # @@ -171,7 +171,7 @@ specified for this file: grep -q -E "${regex_search}" "${file_full_path}" if [ $? -eq 0 ]; then - sed -i -r -e "s%${regex_search}%${regex_replace}%" "${file_full_path}" + $SED -i -r -e "s%${regex_search}%${regex_replace}%" "${file_full_path}" else print_err_msg_exit "\ Specified file (file_full_path) does not contain the searched-for regu- diff --git a/ush/check_ruc_lsm.sh b/ush/check_ruc_lsm.sh index fe22d08ab..8d51cc01b 100644 --- a/ush/check_ruc_lsm.sh +++ b/ush/check_ruc_lsm.sh @@ -30,7 +30,7 @@ function check_ruc_lsm() { # #----------------------------------------------------------------------- # - local scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) + local scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) local scrfunc_fn=$( basename "${scrfunc_fp}" ) local scrfunc_dir=$( dirname "${scrfunc_fp}" ) # @@ -86,7 +86,7 @@ function check_ruc_lsm() { # ruc_lsm_name="lsm_ruc" regex_search="^[ ]*(${ruc_lsm_name})<\/scheme>[ ]*$" - ruc_lsm_name_or_null=$( sed -r -n -e "s/${regex_search}/\1/p" "${ccpp_phys_suite_fp}" ) + ruc_lsm_name_or_null=$( $SED -r -n -e "s/${regex_search}/\1/p" "${ccpp_phys_suite_fp}" ) if [ "${ruc_lsm_name_or_null}" = "${ruc_lsm_name}" ]; then sdf_uses_ruc_lsm="TRUE" diff --git a/ush/cmp_expt_to_baseline.sh b/ush/cmp_expt_to_baseline.sh index 94e6e0c47..b7459593a 100755 --- a/ush/cmp_expt_to_baseline.sh +++ b/ush/cmp_expt_to_baseline.sh @@ -28,7 +28,7 @@ module load nccmp # #----------------------------------------------------------------------- # -scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) scrfunc_fn=$( basename "${scrfunc_fp}" ) scrfunc_dir=$( dirname "${scrfunc_fp}" ) # diff --git a/ush/compare_config_scripts.sh b/ush/compare_config_scripts.sh index 279903ab3..791fa4e34 100644 --- a/ush/compare_config_scripts.sh +++ b/ush/compare_config_scripts.sh @@ -19,7 +19,7 @@ function compare_config_scripts() { # #----------------------------------------------------------------------- # -local scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +local scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) local scrfunc_fn=$( basename "${scrfunc_fp}" ) local scrfunc_dir=$( dirname "${scrfunc_fp}" ) # @@ -65,7 +65,7 @@ local func_name="${FUNCNAME[0]}" #----------------------------------------------------------------------- # var_list_default=$( \ -sed -r \ +$SED -r \ -e "s/^([ ]*)([^ ]+.*)/\2/g" \ -e "/^#.*/d" \ -e "/^$/d" \ @@ -73,7 +73,7 @@ sed -r \ ) var_list_local=$( \ -sed -r \ +$SED -r \ -e "s/^([ ]*)([^ ]+.*)/\2/g" \ -e "/^#.*/d" \ -e "/^$/d" \ @@ -95,7 +95,7 @@ while read crnt_line; do # Note that a variable name will be found only if the equal sign immed- # iately follows the variable name. # - var_name=$( printf "%s" "${crnt_line}" | sed -n -r -e "s/^([^ =\"]*)=.*/\1/p") + var_name=$( printf "%s" "${crnt_line}" | $SED -n -r -e "s/^([^ =\"]*)=.*/\1/p") if [ -z "${var_name}" ]; then diff --git a/ush/config_defaults.sh b/ush/config_defaults.sh index 72c758df4..dd6a2968c 100644 --- a/ush/config_defaults.sh +++ b/ush/config_defaults.sh @@ -46,6 +46,11 @@ RUN_ENVIR="nco" # ACCOUNT: # The account under which to submit jobs to the queue. # +# WORKFLOW_MANAGER: +# The workflow manager to use (e.g. rocoto). This is set to "none" by +# default, but if the machine name is set to a platform that supports +# rocoto, this will be overwritten and set to "rocoto". +# # SCHED: # The job scheduler to use (e.g. slurm). Set this to an empty string in # order for the experiment generation script to set it depending on the @@ -103,6 +108,7 @@ RUN_ENVIR="nco" # MACHINE="BIG_COMPUTER" ACCOUNT="project_name" +WORKFLOW_MANAGER="none" SCHED="" PARTITION_DEFAULT="" QUEUE_DEFAULT="" @@ -113,6 +119,30 @@ QUEUE_FCST="" # #----------------------------------------------------------------------- # +# Set run commands for platforms without a workflow manager. These values +# will be ignored unless WORKFLOW_MANAGER="none". Definitions: +# +# RUN_CMD_UTILS: +# The run command for pre-processing utilities (shave, orog, sfc_climo_gen, etc.) +# Can be left blank for smaller domains, in which case the executables will run +# without MPI. +# +# RUN_CMD_FCST: +# The run command for the model forecast step. This will be appended to the end +# of the variable definitions file, so it can reference other variables. +# +# RUN_CMD_POST: +# The run command for post-processing (UPP). Can be left blank for smaller +# domains, in which case UPP will run without MPI. +# +#----------------------------------------------------------------------- +# +RUN_CMD_UTILS="mpirun -np 1" +RUN_CMD_FCST="mpirun -np \${PE_MEMBER01}" +RUN_CMD_POST="mpirun -np 1" +# +#----------------------------------------------------------------------- +# # Set cron-associated parameters. Definitions: # # USE_CRON_TO_RELAUNCH: diff --git a/ush/create_diag_table_file.sh b/ush/create_diag_table_file.sh index 10ad396ce..3c409031a 100644 --- a/ush/create_diag_table_file.sh +++ b/ush/create_diag_table_file.sh @@ -25,7 +25,7 @@ function create_diag_table_file() { # #----------------------------------------------------------------------- # - local scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) + local scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) local scrfunc_fn=$( basename "${scrfunc_fp}" ) local scrfunc_dir=$( dirname "${scrfunc_fp}" ) # diff --git a/ush/create_model_configure_file.sh b/ush/create_model_configure_file.sh index 23602932b..ccd55d00f 100644 --- a/ush/create_model_configure_file.sh +++ b/ush/create_model_configure_file.sh @@ -25,7 +25,7 @@ function create_model_configure_file() { # #----------------------------------------------------------------------- # - local scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) + local scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) local scrfunc_fn=$( basename "${scrfunc_fp}" ) local scrfunc_dir=$( dirname "${scrfunc_fp}" ) # @@ -99,10 +99,10 @@ run directory (run_dir): # # Set parameters in the model configure file. # - dot_quilting_dot="."${QUILTING,,}"." - dot_print_esmf_dot="."${PRINT_ESMF,,}"." - dot_cpl_dot="."${CPL,,}"." - dot_write_dopost="."${WRITE_DOPOST,,}"." + dot_quilting_dot="."$(echo_lowercase $QUILTING)"." + dot_print_esmf_dot="."$(echo_lowercase $PRINT_ESMF)"." + dot_cpl_dot="."$(echo_lowercase $CPL)"." + dot_write_dopost="."$(echo_lowercase $WRITE_DOPOST)"." # #----------------------------------------------------------------------- # diff --git a/ush/generate_FV3LAM_wflow.sh b/ush/generate_FV3LAM_wflow.sh index e5d2f496e..0ee8eee0b 100755 --- a/ush/generate_FV3LAM_wflow.sh +++ b/ush/generate_FV3LAM_wflow.sh @@ -19,7 +19,11 @@ function generate_FV3LAM_wflow() { # #----------------------------------------------------------------------- # -local scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +if [[ $(uname -s) == Darwin ]]; then + local scrfunc_fp=$( greadlink -f "${BASH_SOURCE[0]}" ) + else + local scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) + fi local scrfunc_fn=$( basename "${scrfunc_fp}" ) local scrfunc_dir=$( dirname "${scrfunc_fp}" ) # @@ -408,7 +412,7 @@ settings="\ # 'sub_hourly_post': ${SUB_HOURLY_POST} 'delta_min': ${DT_SUBHOURLY_POST_MNTS} - 'first_fv3_file_tstr': "000:"`date -d "${DATE_FIRST_CYCL} +${DT_ATMOS} seconds" +%M:%S` + 'first_fv3_file_tstr': "000:"`$DATE_UTIL -d "${DATE_FIRST_CYCL} +${DT_ATMOS} seconds" +%M:%S` " # End of "settings" variable. print_info_msg $VERBOSE " @@ -423,12 +427,13 @@ $settings" # script to generate the experiment's actual XML file from this template # file. # -template_xml_fp="${TEMPLATE_DIR}/${WFLOW_XML_FN}" -$USHDIR/fill_jinja_template.py -q \ - -u "${settings}" \ - -t ${template_xml_fp} \ - -o ${WFLOW_XML_FP} || \ - print_err_msg_exit "\ +if [ "${WORKFLOW_MANAGER}" = "rocoto" ]; then + template_xml_fp="${TEMPLATE_DIR}/${WFLOW_XML_FN}" + $USHDIR/fill_jinja_template.py -q \ + -u "${settings}" \ + -t ${template_xml_fp} \ + -o ${WFLOW_XML_FP} || \ + print_err_msg_exit "\ Call to python script fill_jinja_template.py to create a rocoto workflow XML file from a template file failed. Parameters passed to this script are: @@ -439,6 +444,7 @@ are: Namelist settings specified on command line: settings = $settings" +fi # #----------------------------------------------------------------------- # @@ -468,7 +474,7 @@ if [ "${USE_CRON_TO_RELAUNCH}" = "TRUE" ]; then # # Make a backup copy of the user's crontab file and save it in a file. # - time_stamp=$( date "+%F_%T" ) + time_stamp=$( $DATE_UTIL "+%F_%T" ) crontab_backup_fp="$EXPTDIR/crontab.bak.${time_stamp}" print_info_msg " Copying contents of user cron table to backup file: @@ -485,7 +491,7 @@ Copying contents of user cron table to backup file: # CRONTAB_LINE with backslashes. Do this next. # crontab_line_esc_astr=$( printf "%s" "${CRONTAB_LINE}" | \ - sed -r -e "s%[*]%\\\\*%g" ) + $SED -r -e "s%[*]%\\\\*%g" ) # # In the grep command below, the "^" at the beginning of the string be- # ing passed to grep is a start-of-line anchor while the "$" at the end @@ -549,7 +555,7 @@ if [ "${RUN_ENVIR}" = "nco" ]; then # Resolve the target directory that the FIXam symlink points to and check # that it exists. # - path_resolved=$( readlink -m "$FIXam" ) + path_resolved=$( $READLINK -m "$FIXam" ) if [ ! -d "${path_resolved}" ]; then print_err_msg_exit "\ In order to be able to generate a forecast experiment in NCO mode (i.e. @@ -762,9 +768,9 @@ for (( i=0; i<${num_nml_vars}; i++ )); do mapping="${FV3_NML_VARNAME_TO_FIXam_FILES_MAPPING[$i]}" nml_var_name=$( printf "%s\n" "$mapping" | \ - sed -n -r -e "s/${regex_search}/\1/p" ) + $SED -n -r -e "s/${regex_search}/\1/p" ) FIXam_fn=$( printf "%s\n" "$mapping" | - sed -n -r -e "s/${regex_search}/\2/p" ) + $SED -n -r -e "s/${regex_search}/\2/p" ) fp="\"\"" if [ ! -z "${FIXam_fn}" ]; then @@ -870,9 +876,11 @@ cp_vrfy $USHDIR/${EXPT_CONFIG_FN} $EXPTDIR # #----------------------------------------------------------------------- # -wflow_db_fn="${WFLOW_XML_FN%.xml}.db" -rocotorun_cmd="rocotorun -w ${WFLOW_XML_FN} -d ${wflow_db_fn} -v 10" -rocotostat_cmd="rocotostat -w ${WFLOW_XML_FN} -d ${wflow_db_fn} -v 10" +if [ "${WORKFLOW_MANAGER}" = "rocoto" ]; then + wflow_db_fn="${WFLOW_XML_FN%.xml}.db" + rocotorun_cmd="rocotorun -w ${WFLOW_XML_FN} -d ${wflow_db_fn} -v 10" + rocotostat_cmd="rocotostat -w ${WFLOW_XML_FN} -d ${wflow_db_fn} -v 10" +fi print_info_msg " ======================================================================== @@ -888,25 +896,20 @@ The experiment directory is: > EXPTDIR=\"$EXPTDIR\" " -case "$MACHINE" in - -"CHEYENNE") +# +#----------------------------------------------------------------------- +# +# If rocoto is required, print instructions on how to load and use it +# +#----------------------------------------------------------------------- +# +if [ "${WORKFLOW_MANAGER}" = "rocoto" ]; then print_info_msg "\ To launch the workflow, first ensure that you have a compatible version -of rocoto in your \$PATH. On Cheyenne, version 1.3.1 has been pre-built; -you can load it in your \$PATH with one of the following commands, depending -on your default shell: - -bash: - > export PATH=\${PATH}:/glade/p/ral/jntp/tools/rocoto/rocoto-1.3.1/bin/ - -tcsh: - > setenv PATH \${PATH}:/glade/p/ral/jntp/tools/rocoto/rocoto-1.3.1/bin/ -" - ;; - -*) - print_info_msg "\ +of rocoto available. For most pre-configured platforms, rocoto can be +loaded via a module: + > module load rocoto +For more details on rocoto, see the User's Guide. To launch the workflow, first ensure that you have a compatible version of rocoto loaded. For example, to load version 1.3.1 of rocoto, use @@ -914,11 +917,7 @@ of rocoto loaded. For example, to load version 1.3.1 of rocoto, use (This version has been tested on hera; later versions may also work but have not been tested.) -" - ;; -esac -print_info_msg " To launch the workflow, change location to the experiment directory (EXPTDIR) and issue the rocotrun command, as follows: @@ -939,7 +938,9 @@ Note that: 2) In order for the output of the rocotostat command to be up-to-date, the rocotorun command must be issued immediately before the rocoto- stat command. - +" +fi +print_info_msg " For automatic resubmission of the workflow (say every 3 minutes), the following line can be added to the user's crontab (use \"crontab -e\" to edit the cron table): @@ -991,7 +992,12 @@ set -u # #----------------------------------------------------------------------- # -scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +if [[ $(uname -s) == Darwin ]]; then + command -v greadlink >/dev/null 2>&1 || { echo >&2 "For Darwin-based operating systems (MacOS), the 'greadlink' utility is required to run the UFS SRW Application. Reference the User's Guide for more information about platform requirements. Aborting."; exit 1; } + scrfunc_fp=$( greadlink -f "${BASH_SOURCE[0]}" ) +else + scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +fi scrfunc_fn=$( basename "${scrfunc_fp}" ) scrfunc_dir=$( dirname "${scrfunc_fp}" ) # @@ -1052,7 +1058,7 @@ rm "${tmp_fp}" # ful, move the log file in which the "tee" command saved the output of # the function to the experiment directory. # -if [ $retval -eq 0 ]; then +if [[ $retval == 0 ]]; then mv "${log_fp}" "$exptdir" # # If the call to the generate_FV3LAM_wflow function above was not suc- diff --git a/ush/get_extrn_mdl_file_dir_info.sh b/ush/get_extrn_mdl_file_dir_info.sh index f2224559a..ff7d1bfda 100755 --- a/ush/get_extrn_mdl_file_dir_info.sh +++ b/ush/get_extrn_mdl_file_dir_info.sh @@ -37,7 +37,7 @@ function get_extrn_mdl_file_dir_info() { # #----------------------------------------------------------------------- # - local scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) + local scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) local scrfunc_fn=$( basename "${scrfunc_fp}" ) local scrfunc_dir=$( dirname "${scrfunc_fp}" ) # @@ -209,11 +209,27 @@ fi # #----------------------------------------------------------------------- # +# Declare local function to avoid repetition +# +#----------------------------------------------------------------------- +# +function quit_unless_user_spec_data() { + if [ "${USE_USER_STAGED_EXTRN_FILES}" != "TRUE" ]; then + print_err_msg_exit "\ +The system directory in which to look for external model output files +has not been specified for this external model and machine combination: + extrn_mdl_name = \"${extrn_mdl_name}\" + MACHINE = \"$MACHINE\"" + fi +} +# +#----------------------------------------------------------------------- +# # Check input variables for valid values. # #----------------------------------------------------------------------- # - anl_or_fcst="${anl_or_fcst^^}" + anl_or_fcst=$(echo_uppercase $anl_or_fcst) valid_vals_anl_or_fcst=( "ANL" "FCST" ) check_var_valid_value "anl_or_fcst" "valid_vals_anl_or_fcst" # @@ -234,7 +250,7 @@ fi hh=${cdate_FV3LAM:8:2} yyyymmdd=${cdate_FV3LAM:0:8} - cdate=$( date --utc --date "${yyyymmdd} ${hh} UTC - ${time_offset_hrs} hours" "+%Y%m%d%H" ) + cdate=$( $DATE_UTIL --utc --date "${yyyymmdd} ${hh} UTC - ${time_offset_hrs} hours" "+%Y%m%d%H" ) # #----------------------------------------------------------------------- # @@ -298,7 +314,7 @@ fi # Get the Julian day-of-year of the starting date and time of the exter- # nal model forecast. # - ddd=$( date --utc --date "${yyyy}-${mm}-${dd} ${hh}:${mn} UTC" "+%j" ) + ddd=$( $DATE_UTIL --utc --date "${yyyy}-${mm}-${dd} ${hh}:${mn} UTC" "+%j" ) # # Get the last two digits of the year of the starting date and time of # the external model forecast. @@ -434,12 +450,14 @@ fi ;; *) - print_err_msg_exit "\ + if [ "${USE_USER_STAGED_EXTRN_FILES}" != "TRUE" ]; then + print_err_msg_exit "\ The external model file names (either on disk or in archive files) have not yet been specified for this combination of external model (extrn_mdl_name) and analysis or forecast (anl_or_fcst): extrn_mdl_name = \"${extrn_mdl_name}\" anl_or_fcst = \"${anl_or_fcst}\"" + fi ;; esac @@ -566,12 +584,14 @@ and analysis or forecast (anl_or_fcst): ;; *) - print_err_msg_exit "\ + if [ "${USE_USER_STAGED_EXTRN_FILES}" != "TRUE" ]; then + print_err_msg_exit "\ The external model file names have not yet been specified for this com- bination of external model (extrn_mdl_name) and analysis or forecast (anl_or_fcst): extrn_mdl_name = \"${extrn_mdl_name}\" anl_or_fcst = \"${anl_or_fcst}\"" + fi ;; esac @@ -595,6 +615,7 @@ bination of external model (extrn_mdl_name) and analysis or forecast sysbasedir="${EXTRN_MDL_SYSBASEDIR_LBCS}" fi + sysdir="" case "${extrn_mdl_name}" in # @@ -628,11 +649,7 @@ bination of external model (extrn_mdl_name) and analysis or forecast sysdir="$sysbasedir" ;; *) - print_err_msg_exit "\ -The system directory in which to look for external model output files -has not been specified for this external model and machine combination: - extrn_mdl_name = \"${extrn_mdl_name}\" - MACHINE = \"$MACHINE\"" + quit_unless_user_spec_data ;; esac ;; @@ -665,11 +682,7 @@ has not been specified for this external model and machine combination: sysdir="$sysbasedir" ;; *) - print_err_msg_exit "\ -The system directory in which to look for external model output files -has not been specified for this external model and machine combination: - extrn_mdl_name = \"${extrn_mdl_name}\" - MACHINE = \"$MACHINE\"" + quit_unless_user_spec_data ;; esac ;; @@ -699,11 +712,7 @@ has not been specified for this external model and machine combination: sysdir="$sysbasedir" ;; *) - print_err_msg_exit "\ -The system directory in which to look for external model output files -has not been specified for this external model and machine combination: - extrn_mdl_name = \"${extrn_mdl_name}\" - MACHINE = \"$MACHINE\"" + quit_unless_user_spec_data ;; esac ;; @@ -733,11 +742,7 @@ has not been specified for this external model and machine combination: sysdir="$sysbasedir" ;; *) - print_err_msg_exit "\ -The system directory in which to look for external model output files -has not been specified for this external model and machine combination: - extrn_mdl_name = \"${extrn_mdl_name}\" - MACHINE = \"$MACHINE\"" + quit_unless_user_spec_data ;; esac ;; @@ -766,22 +771,14 @@ has not been specified for this external model and machine combination: sysdir="$sysbasedir" ;; *) - print_err_msg_exit "\ -The system directory in which to look for external model output files -has not been specified for this external model and machine combination: - extrn_mdl_name = \"${extrn_mdl_name}\" - MACHINE = \"$MACHINE\"" + quit_unless_user_spec_data ;; esac ;; *) - print_err_msg_exit "\ -The system directory in which to look for external model output files -has not been specified for this external model: - extrn_mdl_name = \"${extrn_mdl_name}\"" - + quit_unless_user_spec_data esac # #----------------------------------------------------------------------- diff --git a/ush/launch_FV3LAM_wflow.sh b/ush/launch_FV3LAM_wflow.sh index 453db87d3..72cdd74c7 100755 --- a/ush/launch_FV3LAM_wflow.sh +++ b/ush/launch_FV3LAM_wflow.sh @@ -17,7 +17,12 @@ set -u # #----------------------------------------------------------------------- # -scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +if [[ $(uname -s) == Darwin ]]; then + command -v greadlink >/dev/null 2>&1 || { echo >&2 "For Darwin-based operating systems (MacOS), the 'greadlink' utility is required to run the UFS SRW Application. Reference the User's Guide for more information about platform requirements. Aborting."; exit 1; } + scrfunc_fp=$( greadlink -f "${BASH_SOURCE[0]}" ) +else + scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +fi scrfunc_fn=$( basename "${scrfunc_fp}" ) scrfunc_dir=$( dirname "${scrfunc_fp}" ) # @@ -65,7 +70,12 @@ scrfunc_dir=$( dirname "${scrfunc_fp}" ) #----------------------------------------------------------------------- # exptdir=$( dirname "$0" ) -exptdir=$( readlink -f "$exptdir" ) +if [[ $(uname -s) == Darwin ]]; then + command -v greadlink >/dev/null 2>&1 || { echo >&2 "For Darwin-based operating systems (MacOS), the 'greadlink' utility is required to run the UFS SRW Application. Reference the User's Guide for more information about platform requirements. Aborting."; exit 1; } + exptdir=$( greadlink -f "$exptdir" ) +else + exptdir=$( readlink -f "$exptdir" ) +fi # #----------------------------------------------------------------------- # @@ -395,7 +405,7 @@ launch script for this experiment: # CRONTAB_LINE with backslashes. Do this next. # crontab_line_esc_astr=$( printf "%s" "${CRONTAB_LINE}" | \ - sed -r -e "s%[*]%\\\\*%g" ) + $SED -r -e "s%[*]%\\\\*%g" ) # # In the string passed to the grep command below, we use the line start # and line end anchors ("^" and "$", respectively) to ensure that we on- diff --git a/ush/link_fix.sh b/ush/link_fix.sh index e9a641d96..34bcd5a53 100755 --- a/ush/link_fix.sh +++ b/ush/link_fix.sh @@ -26,7 +26,7 @@ function link_fix() { # #----------------------------------------------------------------------- # - local scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) + local scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) local scrfunc_fn=$( basename "${scrfunc_fp}" ) local scrfunc_dir=$( dirname "${scrfunc_fp}" ) # @@ -289,7 +289,7 @@ Creating links in the FIXLAM directory to the grid files..." fn=$( basename $fp ) - res=$( printf "%s" $fn | sed -n -r -e "s/^C([0-9]*).*/\1/p" ) + res=$( printf "%s" $fn | $SED -n -r -e "s/^C([0-9]*).*/\1/p" ) if [ -z $res ]; then print_err_msg_exit "\ The resolution could not be extracted from the current file's name. The diff --git a/ush/load_modules_run_task.sh b/ush/load_modules_run_task.sh index 8b91ef0cf..411f0bdb1 100755 --- a/ush/load_modules_run_task.sh +++ b/ush/load_modules_run_task.sh @@ -27,7 +27,7 @@ # #----------------------------------------------------------------------- # -scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) scrfunc_fn=$( basename "${scrfunc_fp}" ) scrfunc_dir=$( dirname "${scrfunc_fp}" ) # @@ -152,7 +152,7 @@ jjob_fp="$2" # #----------------------------------------------------------------------- # -machine=${MACHINE,,} +machine=$(echo_lowercase $MACHINE) env_fn="build_${machine}_${COMPILER}.env" env_fp="${SR_WX_APP_TOP_DIR}/env/${env_fn}" source "${env_fp}" || print_err_msg_exit "\ @@ -191,89 +191,6 @@ modulefile_name="${task_name}" default_modules_dir="$HOMErrfs/modulefiles" default_modulefile_name="${machine}.default" use_default_modulefile=0 -####### -####### The following lines (199-276) can be removed once we confirm -####### that the new method of setting environment variables and loading -####### modules will remain permanent. -####### -# -#----------------------------------------------------------------------- -# -# This comment needs to be updated: -# -# Use the "readlink" command to resolve the full path to the module file -# and then verify that the file exists. This is not necessary for most -# tasks, but for the run_fcst task, when CCPP is enabled, the module -# file in the modules directory is not a regular file but a symlink to a -# file in the ufs_weather_model external repo. This latter target file -# will exist only if the forecast model code has already been built. -# Thus, we now check to make sure that the module file exits. -# -#----------------------------------------------------------------------- -# -#if [ "${machine}" = "unknown" ]; then -# -# This if-statement allows for a graceful exit in the case in which module -# files are not needed for the task. -# This is not currently used but reserved for future development. -# -# print_info_msg " -#Module files are not needed for this task (task_name) and machine (machine): -# task_name = \"${task_name}\" -# machine = \"${machine}\"" - -#else - -# modulefile_path=$( readlink -f "${modules_dir}/${modulefile_name}" ) - -# if [ ! -f "${modulefile_path}" ]; then - -# default_modulefile_path=$( readlink -f "${default_modules_dir}/${default_modulefile_name}" ) -# if [ -f "${default_modulefile_path}" ]; then -# -# If the task-specific modulefile does not exist but a default one does, -# use it! -# -# print_info_msg "$VERBOSE" " -#A task-specific modulefile (modulefile_path) does not exist for this task -#(task_name) and machine (machine) combination: -# task_name = \"${task_name}\" -# machine = \"${machine}\" -# modulefile_path = \"${modulefile_path}\" -#Will attempt to use the default modulefile (default_modulefile_path): -# default_modulefile_path = \"${default_modulefile_path}\"" -# -# modules_dir="${default_modules_dir}" -# use_default_modulefile=1 -# -# elif [ "${task_name}" = "${MAKE_OROG_TN}" ] || \ -# [ "${task_name}" = "${MAKE_SFC_CLIMO_TN}" ] || \ -# [ "${task_name}" = "${MAKE_ICS_TN}" ] || \ -# [ "${task_name}" = "${MAKE_LBCS_TN}" ] || \ -# [ "${task_name}" = "${RUN_FCST_TN}" ]; then -# -# print_err_msg_exit "\ -#The target (modulefile_path) of the symlink (modulefile_name) in the task -#modules directory (modules_dir) that points to module file for this task -#(task_name) does not exist: -# task_name = \"${task_name}\" -# modulefile_name = \"${modulefile_name}\" -# modules_dir = \"${modules_dir}\" -# modulefile_path = \"${modulefile_path}\" -#This is likely because the forecast model code has not yet been built." -# -# else -# -# print_err_msg_exit "\ -#The module file (modulefile_path) specified for this task (task_name) -#does not exist: -# task_name = \"${task_name}\" -# modulefile_path = \"${modulefile_path}\" -# machine = \"${machine}\"" -# -# fi -# -# fi # #----------------------------------------------------------------------- # diff --git a/ush/make_grid_mosaic_file.sh b/ush/make_grid_mosaic_file.sh index 0026cb86a..8a81feb14 100644 --- a/ush/make_grid_mosaic_file.sh +++ b/ush/make_grid_mosaic_file.sh @@ -27,7 +27,7 @@ function make_grid_mosaic_file() { # #----------------------------------------------------------------------- # - local scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) + local scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) local scrfunc_fn=$( basename "${scrfunc_fp}" ) local scrfunc_dir=$( dirname "${scrfunc_fp}" ) # diff --git a/ush/set_FV3nml_sfc_climo_filenames.sh b/ush/set_FV3nml_sfc_climo_filenames.sh index 2d44a9d0c..13f2ecdf5 100644 --- a/ush/set_FV3nml_sfc_climo_filenames.sh +++ b/ush/set_FV3nml_sfc_climo_filenames.sh @@ -30,7 +30,7 @@ function set_FV3nml_sfc_climo_filenames() { # #----------------------------------------------------------------------- # - local scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) + local scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) local scrfunc_fn=$( basename "${scrfunc_fp}" ) local scrfunc_dir=$( dirname "${scrfunc_fp}" ) # @@ -145,9 +145,9 @@ for (( i=0; i<${num_nml_vars}; i++ )); do mapping="${FV3_NML_VARNAME_TO_SFC_CLIMO_FIELD_MAPPING[$i]}" nml_var_name=$( printf "%s\n" "$mapping" | \ - sed -n -r -e "s/${regex_search}/\1/p" ) + $SED -n -r -e "s/${regex_search}/\1/p" ) sfc_climo_field_name=$( printf "%s\n" "$mapping" | - sed -n -r -e "s/${regex_search}/\2/p" ) + $SED -n -r -e "s/${regex_search}/\2/p" ) # # Check that the surface climatology field associated with the current # namelist variable is valid. diff --git a/ush/set_FV3nml_stoch_params.sh b/ush/set_FV3nml_stoch_params.sh index a6d4c6dfd..cfe115d52 100644 --- a/ush/set_FV3nml_stoch_params.sh +++ b/ush/set_FV3nml_stoch_params.sh @@ -32,7 +32,7 @@ function set_FV3nml_stoch_params() { # #----------------------------------------------------------------------- # - local scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) + local scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) local scrfunc_fn=$( basename "${scrfunc_fp}" ) local scrfunc_dir=$( dirname "${scrfunc_fp}" ) # diff --git a/ush/set_cycle_dates.sh b/ush/set_cycle_dates.sh index b8d406c12..c4b3adfbf 100644 --- a/ush/set_cycle_dates.sh +++ b/ush/set_cycle_dates.sh @@ -30,7 +30,7 @@ function set_cycle_dates() { # #----------------------------------------------------------------------- # - local scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) + local scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) local scrfunc_fn=$( basename "${scrfunc_fp}" ) local scrfunc_dir=$( dirname "${scrfunc_fp}" ) # @@ -106,7 +106,7 @@ End date (date_end) must be at or after start date (date_start): date_crnt="${date_start}" while [ "${date_crnt}" -le "${date_end}" ]; do all_cdates+=( $( printf "%s " ${cycle_hrs[@]/#/${date_crnt}} ) ) - date_crnt=$( date -d "${date_crnt} + 1 days" +%Y%m%d ) + date_crnt=$( $DATE_UTIL -d "${date_crnt} + 1 days" +%Y%m%d ) done # #----------------------------------------------------------------------- diff --git a/ush/set_extrn_mdl_params.sh b/ush/set_extrn_mdl_params.sh index 72855d9fb..c40e3a128 100644 --- a/ush/set_extrn_mdl_params.sh +++ b/ush/set_extrn_mdl_params.sh @@ -17,7 +17,7 @@ function set_extrn_mdl_params() { # #----------------------------------------------------------------------- # -local scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +local scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) local scrfunc_fn=$( basename "${scrfunc_fp}" ) local scrfunc_dir=$( dirname "${scrfunc_fp}" ) # diff --git a/ush/set_gridparams_ESGgrid.sh b/ush/set_gridparams_ESGgrid.sh index 91ec8b83d..41efbfe58 100644 --- a/ush/set_gridparams_ESGgrid.sh +++ b/ush/set_gridparams_ESGgrid.sh @@ -26,7 +26,7 @@ function set_gridparams_ESGgrid() { # #----------------------------------------------------------------------- # - local scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) + local scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) local scrfunc_fn=$( basename "${scrfunc_fp}" ) local scrfunc_dir=$( dirname "${scrfunc_fp}" ) # diff --git a/ush/set_gridparams_GFDLgrid.sh b/ush/set_gridparams_GFDLgrid.sh index 7d47affb1..a76b9a58a 100644 --- a/ush/set_gridparams_GFDLgrid.sh +++ b/ush/set_gridparams_GFDLgrid.sh @@ -26,7 +26,7 @@ function set_gridparams_GFDLgrid() { # #----------------------------------------------------------------------- # - local scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) + local scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) local scrfunc_fn=$( basename "${scrfunc_fp}" ) local scrfunc_dir=$( dirname "${scrfunc_fp}" ) # @@ -425,8 +425,8 @@ AFTER adjustments are: nx_of_t6_on_t6sg=$(( 2*nx_of_t6_on_t6g )) ny_of_t6_on_t6sg=$(( 2*ny_of_t6_on_t6g )) - prime_factors_nx_of_t7_on_t7g=$( factor ${nx_of_t7_on_t7g} | sed -r -e 's/^[0-9]+: (.*)/\1/' ) - prime_factors_ny_of_t7_on_t7g=$( factor ${ny_of_t7_on_t7g} | sed -r -e 's/^[0-9]+: (.*)/\1/' ) + prime_factors_nx_of_t7_on_t7g=$( factor ${nx_of_t7_on_t7g} | $SED -r -e 's/^[0-9]+: (.*)/\1/' ) + prime_factors_ny_of_t7_on_t7g=$( factor ${ny_of_t7_on_t7g} | $SED -r -e 's/^[0-9]+: (.*)/\1/' ) print_info_msg "$VERBOSE" " The number of cells in the two horizontal directions (x and y) on the diff --git a/ush/set_ozone_param.sh b/ush/set_ozone_param.sh index 5fb90f839..c85749279 100644 --- a/ush/set_ozone_param.sh +++ b/ush/set_ozone_param.sh @@ -44,7 +44,7 @@ function set_ozone_param() { # #----------------------------------------------------------------------- # - local scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) + local scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) local scrfunc_fn=$( basename "${scrfunc_fp}" ) local scrfunc_dir=$( dirname "${scrfunc_fp}" ) # @@ -130,7 +130,7 @@ function set_ozone_param() { #----------------------------------------------------------------------- # regex_search="^[ ]*(ozphys.*)<\/scheme>[ ]*$" - ozone_param=$( sed -r -n -e "s/${regex_search}/\1/p" "${ccpp_phys_suite_fp}" ) + ozone_param=$( $SED -r -n -e "s/${regex_search}/\1/p" "${ccpp_phys_suite_fp}" ) if [ "${ozone_param}" = "ozphys_2015" ]; then fixgsm_ozone_fn="ozprdlos_2015_new_sbuvO3_tclm15_nuchem.f77" @@ -175,11 +175,11 @@ num_symlinks=${#CYCLEDIR_LINKS_TO_FIXam_FILES_MAPPING[@]} for (( i=0; i<${num_symlinks}; i++ )); do mapping="${CYCLEDIR_LINKS_TO_FIXam_FILES_MAPPING[$i]}" symlink=$( printf "%s\n" "$mapping" | \ - sed -n -r -e "s/${regex_search}/\1/p" ) + $SED -n -r -e "s/${regex_search}/\1/p" ) if [ "$symlink" = "${ozone_symlink}" ]; then regex_search="^[ ]*([^| ]+[ ]*)[|][ ]*([^| ]*)[ ]*$" mapping_ozone=$( printf "%s\n" "$mapping" | \ - sed -n -r -e "s/${regex_search}/\1/p" ) + $SED -n -r -e "s/${regex_search}/\1/p" ) mapping_ozone="${mapping_ozone}| ${fixgsm_ozone_fn}" CYCLEDIR_LINKS_TO_FIXam_FILES_MAPPING[$i]="${mapping_ozone}" fixgsm_ozone_fn_is_set="TRUE" diff --git a/ush/set_predef_grid_params.sh b/ush/set_predef_grid_params.sh index 698a7e2a8..aca2a26e3 100644 --- a/ush/set_predef_grid_params.sh +++ b/ush/set_predef_grid_params.sh @@ -25,7 +25,7 @@ function set_predef_grid_params() { # #----------------------------------------------------------------------- # -local scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +local scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) local scrfunc_fn=$( basename "${scrfunc_fp}" ) local scrfunc_dir=$( dirname "${scrfunc_fp}" ) # diff --git a/ush/set_thompson_mp_fix_files.sh b/ush/set_thompson_mp_fix_files.sh index 558333435..667ad522f 100644 --- a/ush/set_thompson_mp_fix_files.sh +++ b/ush/set_thompson_mp_fix_files.sh @@ -32,7 +32,7 @@ function set_thompson_mp_fix_files() { # #----------------------------------------------------------------------- # - local scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) + local scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) local scrfunc_fn=$( basename "${scrfunc_fp}" ) local scrfunc_dir=$( dirname "${scrfunc_fp}" ) # @@ -93,7 +93,7 @@ function set_thompson_mp_fix_files() { # thompson_mp_name="mp_thompson" regex_search="^[ ]*(${thompson_mp_name})<\/scheme>[ ]*$" - thompson_mp_name_or_null=$( sed -r -n -e "s/${regex_search}/\1/p" "${ccpp_phys_suite_fp}" ) + thompson_mp_name_or_null=$( $SED -r -n -e "s/${regex_search}/\1/p" "${ccpp_phys_suite_fp}" ) if [ "${thompson_mp_name_or_null}" = "${thompson_mp_name}" ]; then sdf_uses_thompson_mp="TRUE" diff --git a/ush/setup.sh b/ush/setup.sh index bc11c67b4..96b1ef071 100755 --- a/ush/setup.sh +++ b/ush/setup.sh @@ -23,7 +23,7 @@ function setup() { # #----------------------------------------------------------------------- # -local scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +local scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) local scrfunc_fn=$( basename "${scrfunc_fp}" ) local scrfunc_dir=$( dirname "${scrfunc_fp}" ) # @@ -140,7 +140,7 @@ check_var_valid_value "VERBOSE" "valid_vals_VERBOSE" # Set VERBOSE to either "TRUE" or "FALSE" so we don't have to consider # other valid values later on. # -VERBOSE=${VERBOSE^^} +VERBOSE=$(echo_uppercase $VERBOSE) if [ "$VERBOSE" = "TRUE" ] || \ [ "$VERBOSE" = "YES" ]; then VERBOSE="TRUE" @@ -160,7 +160,7 @@ check_var_valid_value "USE_CRON_TO_RELAUNCH" "valid_vals_USE_CRON_TO_RELAUNCH" # Set USE_CRON_TO_RELAUNCH to either "TRUE" or "FALSE" so we don't have to consider # other valid values later on. # -USE_CRON_TO_RELAUNCH=${USE_CRON_TO_RELAUNCH^^} +USE_CRON_TO_RELAUNCH=$(echo_uppercase $USE_CRON_TO_RELAUNCH) if [ "${USE_CRON_TO_RELAUNCH}" = "TRUE" ] || \ [ "${USE_CRON_TO_RELAUNCH}" = "YES" ]; then USE_CRON_TO_RELAUNCH="TRUE" @@ -180,7 +180,7 @@ check_var_valid_value "RUN_TASK_MAKE_GRID" "valid_vals_RUN_TASK_MAKE_GRID" # Set RUN_TASK_MAKE_GRID to either "TRUE" or "FALSE" so we don't have to # consider other valid values later on. # -RUN_TASK_MAKE_GRID=${RUN_TASK_MAKE_GRID^^} +RUN_TASK_MAKE_GRID=$(echo_uppercase $RUN_TASK_MAKE_GRID) if [ "${RUN_TASK_MAKE_GRID}" = "TRUE" ] || \ [ "${RUN_TASK_MAKE_GRID}" = "YES" ]; then RUN_TASK_MAKE_GRID="TRUE" @@ -200,7 +200,7 @@ check_var_valid_value "RUN_TASK_MAKE_OROG" "valid_vals_RUN_TASK_MAKE_OROG" # Set RUN_TASK_MAKE_OROG to either "TRUE" or "FALSE" so we don't have to # consider other valid values later on. # -RUN_TASK_MAKE_OROG=${RUN_TASK_MAKE_OROG^^} +RUN_TASK_MAKE_OROG=$(echo_uppercase $RUN_TASK_MAKE_OROG) if [ "${RUN_TASK_MAKE_OROG}" = "TRUE" ] || \ [ "${RUN_TASK_MAKE_OROG}" = "YES" ]; then RUN_TASK_MAKE_OROG="TRUE" @@ -221,7 +221,7 @@ check_var_valid_value \ # Set RUN_TASK_MAKE_SFC_CLIMO to either "TRUE" or "FALSE" so we don't # have to consider other valid values later on. # -RUN_TASK_MAKE_SFC_CLIMO=${RUN_TASK_MAKE_SFC_CLIMO^^} +RUN_TASK_MAKE_SFC_CLIMO=$(echo_uppercase $RUN_TASK_MAKE_SFC_CLIMO) if [ "${RUN_TASK_MAKE_SFC_CLIMO}" = "TRUE" ] || \ [ "${RUN_TASK_MAKE_SFC_CLIMO}" = "YES" ]; then RUN_TASK_MAKE_SFC_CLIMO="TRUE" @@ -242,7 +242,7 @@ check_var_valid_value \ # Set RUN_TASK_RUN_POST to either "TRUE" or "FALSE" so we don't # have to consider other valid values later on. # -RUN_TASK_RUN_POST=${RUN_TASK_RUN_POST^^} +RUN_TASK_RUN_POST=$(echo_uppercase $RUN_TASK_RUN_POST) if [ "${RUN_TASK_RUN_POST}" = "TRUE" ] || \ [ "${RUN_TASK_RUN_POST}" = "YES" ]; then RUN_TASK_RUN_POST="TRUE" @@ -262,7 +262,7 @@ check_var_valid_value "RUN_TASK_VX_GRIDSTAT" "valid_vals_RUN_TASK_VX_GRIDSTAT" # Set RUN_TASK_VX_GRIDSTAT to either "TRUE" or "FALSE" so we don't have to # consider other valid values later on. # -RUN_TASK_VX_GRIDSTAT=${RUN_TASK_VX_GRIDSTAT^^} +RUN_TASK_VX_GRIDSTAT=$(echo_uppercase $RUN_TASK_VX_GRIDSTAT) if [ "${RUN_TASK_VX_GRIDSTAT}" = "TRUE" ] || \ [ "${RUN_TASK_VX_GRIDSTAT}" = "YES" ]; then RUN_TASK_VX_GRIDSTAT="TRUE" @@ -282,7 +282,7 @@ check_var_valid_value "RUN_TASK_VX_POINTSTAT" "valid_vals_RUN_TASK_VX_POINTSTAT" # Set RUN_TASK_VX_POINTSTAT to either "TRUE" or "FALSE" so we don't have to # consider other valid values later on. # -RUN_TASK_VX_POINTSTAT=${RUN_TASK_VX_POINTSTAT^^} +RUN_TASK_VX_POINTSTAT=$(echo_uppercase $RUN_TASK_VX_POINTSTAT) if [ "${RUN_TASK_VX_POINTSTAT}" = "TRUE" ] || \ [ "${RUN_TASK_VX_POINTSTAT}" = "YES" ]; then RUN_TASK_VX_POINTSTAT="TRUE" @@ -303,7 +303,7 @@ check_var_valid_value "RUN_TASK_VX_ENSGRID" "valid_vals_RUN_TASK_VX_ENSGRID" # Set RUN_TASK_VX_ENSGRID to either "TRUE" or "FALSE" so we don't have to # consider other valid values later on. # -RUN_TASK_VX_ENSGRID=${RUN_TASK_VX_ENSGRID^^} +RUN_TASK_VX_ENSGRID=$(echo_uppercase $RUN_TASK_VX_ENSGRID) if [ "${RUN_TASK_VX_ENSGRID}" = "TRUE" ] || \ [ "${RUN_TASK_VX_ENSGRID}" = "YES" ]; then RUN_TASK_VX_ENSGRID="TRUE" @@ -325,7 +325,7 @@ check_var_valid_value "RUN_TASK_VX_ENSPOINT" "valid_vals_RUN_TASK_VX_ENSPOINT" # Set RUN_TASK_VX_ENSPOINT to either "TRUE" or "FALSE" so we don't have to # consider other valid values later on. # -RUN_TASK_VX_ENSPOINT=${RUN_TASK_VX_ENSPOINT^^} +RUN_TASK_VX_ENSPOINT=$(echo_uppercase $RUN_TASK_VX_ENSPOINT) if [ "${RUN_TASK_VX_ENSPOINT}" = "TRUE" ] || \ [ "${RUN_TASK_VX_ENSPOINT}" = "YES" ]; then RUN_TASK_VX_ENSPOINT="TRUE" @@ -345,7 +345,7 @@ check_var_valid_value "DO_SHUM" "valid_vals_DO_SHUM" # Set DO_SHUM to either "TRUE" or "FALSE" so we don't # have to consider other valid values later on. # -DO_SHUM=${DO_SHUM^^} +DO_SHUM=$(echo_uppercase $DO_SHUM) if [ "${DO_SHUM}" = "TRUE" ] || \ [ "${DO_SHUM}" = "YES" ]; then DO_SHUM="TRUE" @@ -365,7 +365,7 @@ check_var_valid_value "DO_SPPT" "valid_vals_DO_SPPT" # Set DO_SPPT to either "TRUE" or "FALSE" so we don't # have to consider other valid values later on. # -DO_SPPT=${DO_SPPT^^} +DO_SPPT=$(echo_uppercase $DO_SPPT) if [ "${DO_SPPT}" = "TRUE" ] || \ [ "${DO_SPPT}" = "YES" ]; then DO_SPPT="TRUE" @@ -385,7 +385,7 @@ check_var_valid_value "DO_SKEB" "valid_vals_DO_SKEB" # Set DO_SKEB to either "TRUE" or "FALSE" so we don't # have to consider other valid values later on. # -DO_SKEB=${DO_SKEB^^} +DO_SKEB=$(echo_uppercase $DO_SKEB) if [ "${DO_SKEB}" = "TRUE" ] || \ [ "${DO_SKEB}" = "YES" ]; then DO_SKEB="TRUE" @@ -405,7 +405,7 @@ check_var_valid_value "DO_SPP" "valid_vals_DO_SPP" # Set DO_SPP to either "TRUE" or "FALSE" so we don't # have to consider other valid values later on. # -DO_SPP=${DO_SPP^^} +DO_SPP=$(echo_uppercase $DO_SPP) if [ "${DO_SPP}" = "TRUE" ] || \ [ "${DO_SPP}" = "YES" ]; then DO_SPP="TRUE" @@ -458,7 +458,7 @@ check_var_valid_value "USE_FVCOM" "valid_vals_USE_FVCOM" # Set USE_FVCOM to either "TRUE" or "FALSE" so we don't have to consider # other valid values later on. # -USE_FVCOM=${USE_FVCOM^^} +USE_FVCOM=$(echo_uppercase $USE_FVCOM) if [ "$USE_FVCOM" = "TRUE" ] || \ [ "$USE_FVCOM" = "YES" ]; then USE_FVCOM="TRUE" @@ -479,7 +479,7 @@ check_var_valid_value "SUB_HOURLY_POST" "valid_vals_SUB_HOURLY_POST" # Set SUB_HOURLY_POST to either "TRUE" or "FALSE" so we don't have to consider # other valid values later on. # -SUB_HOURLY_POST=${SUB_HOURLY_POST^^} +SUB_HOURLY_POST=$(echo_uppercase $SUB_HOURLY_POST) if [ "${SUB_HOURLY_POST}" = "TRUE" ] || \ [ "${SUB_HOURLY_POST}" = "YES" ]; then SUB_HOURLY_POST="TRUE" @@ -503,7 +503,7 @@ check_var_valid_value "DOT_OR_USCORE" "valid_vals_DOT_OR_USCORE" # #----------------------------------------------------------------------- # -MACHINE=$( printf "%s" "$MACHINE" | sed -e 's/\(.*\)/\U\1/' ) +MACHINE=$( printf "%s" "$MACHINE" | $SED -e 's/\(.*\)/\U\1/' ) check_var_valid_value "MACHINE" "valid_vals_MACHINE" # #----------------------------------------------------------------------- @@ -520,10 +520,11 @@ check_var_valid_value "MACHINE" "valid_vals_MACHINE" #----------------------------------------------------------------------- # RELATIVE_LINK_FLAG="" - -case "$MACHINE" in +NCORES_PER_NODE="2" # Need some arbitrary default value to avoid division by zero errors +case $MACHINE in "WCOSS_CRAY") + WORKFLOW_MANAGER="rocoto" NCORES_PER_NODE="24" SCHED="lsfcray" QUEUE_DEFAULT=${QUEUE_DEFAULT:-"dev"} @@ -534,6 +535,7 @@ case "$MACHINE" in ;; "WCOSS_DELL_P3") + WORKFLOW_MANAGER="rocoto" NCORES_PER_NODE=24 SCHED="lsf" QUEUE_DEFAULT=${QUEUE_DEFAULT:-"dev"} @@ -544,6 +546,7 @@ case "$MACHINE" in ;; "HERA") + WORKFLOW_MANAGER="rocoto" NCORES_PER_NODE=40 SCHED=${SCHED:-"slurm"} PARTITION_DEFAULT=${PARTITION_DEFAULT:-"hera"} @@ -557,6 +560,7 @@ case "$MACHINE" in ;; "ORION") + WORKFLOW_MANAGER="rocoto" NCORES_PER_NODE=40 SCHED=${SCHED:-"slurm"} PARTITION_DEFAULT=${PARTITION_DEFAULT:-"orion"} @@ -570,6 +574,7 @@ case "$MACHINE" in ;; "JET") + WORKFLOW_MANAGER="rocoto" NCORES_PER_NODE=24 SCHED=${SCHED:-"slurm"} PARTITION_DEFAULT=${PARTITION_DEFAULT:-"sjet,vjet,kjet,xjet"} @@ -583,6 +588,7 @@ case "$MACHINE" in ;; "ODIN") + WORKFLOW_MANAGER="rocoto" NCORES_PER_NODE=24 SCHED=${SCHED:-"slurm"} PARTITION_DEFAULT=${PARTITION_DEFAULT:-"workq"} @@ -596,6 +602,7 @@ case "$MACHINE" in ;; "CHEYENNE") + WORKFLOW_MANAGER="rocoto" NCORES_PER_NODE=36 SCHED=${SCHED:-"pbspro"} QUEUE_DEFAULT=${QUEUE_DEFAULT:-"regular"} @@ -606,6 +613,7 @@ case "$MACHINE" in ;; "STAMPEDE") + WORKFLOW_MANAGER="rocoto" NCORES_PER_NODE=68 SCHED="slurm" PARTITION_DEFAULT=${PARTITION_DEFAULT:-"normal"} @@ -618,6 +626,16 @@ case "$MACHINE" in RELATIVE_LINK_FLAG="--relative" ;; + "MACOS") + WORKFLOW_MANAGER="none" + SCHED="none" + ;; + + "LINUX") + WORKFLOW_MANAGER="none" + SCHED="none" + ;; + esac # #----------------------------------------------------------------------- @@ -635,20 +653,27 @@ PPN_RUN_FCST=${PPN_RUN_FCST:-${PPN_RUN_FCST_OPT}} # #----------------------------------------------------------------------- # -SCHED="${SCHED,,}" +SCHED=$(echo_lowercase $SCHED) check_var_valid_value "SCHED" "valid_vals_SCHED" # #----------------------------------------------------------------------- # -# Verify that the ACCOUNT variable is not empty. If it is, print out an -# error message and exit. +# If we are using a workflow manager, run some checks. First, +# verify that the ACCOUNT variable is not empty. Second, ensure that the +# custom RUN_CMD variables are not set. # #----------------------------------------------------------------------- # -if [ -z "$ACCOUNT" ]; then - print_err_msg_exit "\ -The variable ACCOUNT cannot be empty: - ACCOUNT = \"$ACCOUNT\"" +if [ "$WORKFLOW_MANAGER" != "none" ]; then + if [ -z "$ACCOUNT" ]; then + print_err_msg_exit "\ +The variable ACCOUNT cannot be empty if you are using a workflow manager: + ACCOUNT = \"$ACCOUNT\" + WORKFLOW_MANAGER = \"$WORKFLOW_MANAGER\"" + fi + RUN_CMD_UTILS="" + RUN_CMD_FCST="" + RUN_CMD_POST="" fi # #----------------------------------------------------------------------- @@ -753,7 +778,7 @@ fi #----------------------------------------------------------------------- # DATE_OR_NULL=$( printf "%s" "${DATE_FIRST_CYCL}" | \ - sed -n -r -e "s/^([0-9]{8})$/\1/p" ) + $SED -n -r -e "s/^([0-9]{8})$/\1/p" ) if [ -z "${DATE_OR_NULL}" ]; then print_err_msg_exit "\ DATE_FIRST_CYCL must be a string consisting of exactly 8 digits of the @@ -763,7 +788,7 @@ month, and DD is the 2-digit day-of-month. fi DATE_OR_NULL=$( printf "%s" "${DATE_LAST_CYCL}" | \ - sed -n -r -e "s/^([0-9]{8})$/\1/p" ) + $SED -n -r -e "s/^([0-9]{8})$/\1/p" ) if [ -z "${DATE_OR_NULL}" ]; then print_err_msg_exit "\ DATE_LAST_CYCL must be a string consisting of exactly 8 digits of the @@ -785,7 +810,7 @@ CYCL_HRS_str="( $CYCL_HRS_str)" i=0 for CYCL in "${CYCL_HRS[@]}"; do - CYCL_OR_NULL=$( printf "%s" "$CYCL" | sed -n -r -e "s/^([0-9]{2})$/\1/p" ) + CYCL_OR_NULL=$( printf "%s" "$CYCL" | $SED -n -r -e "s/^([0-9]{2})$/\1/p" ) if [ -z "${CYCL_OR_NULL}" ]; then print_err_msg_exit "\ @@ -955,7 +980,8 @@ case "$MACHINE" in ;; *) - print_err_msg_exit "\ + if [ -z "$FIXgsm" -o -z "$TOPO_DIR" -o -z "$SFC_CLIMO_INPUT_DIR" ]; then + print_err_msg_exit "\ One or more fix file directories have not been specified for this machine: MACHINE = \"$MACHINE\" FIXgsm = \"${FIXgsm:-\"\"} @@ -963,6 +989,7 @@ One or more fix file directories have not been specified for this machine: SFC_CLIMO_INPUT_DIR = \"${SFC_CLIMO_INPUT_DIR:-\"\"} FIXLAM_NCO_BASEDIR = \"${FIXLAM_NCO_BASEDIR:-\"\"} You can specify the missing location(s) in config.sh" + fi ;; esac @@ -979,7 +1006,7 @@ esac # #----------------------------------------------------------------------- # -mng_extrns_cfg_fn=$( readlink -f "${SR_WX_APP_TOP_DIR}/Externals.cfg" ) +mng_extrns_cfg_fn=$( $READLINK -f "${SR_WX_APP_TOP_DIR}/Externals.cfg" ) property_name="local_path" # # Get the base directory of the FV3 forecast model code. @@ -1051,7 +1078,7 @@ check_var_valid_value \ # Set USE_CUSTOM_POST_CONFIG_FILE to either "TRUE" or "FALSE" so we don't # have to consider other valid values later on. # -USE_CUSTOM_POST_CONFIG_FILE=${USE_CUSTOM_POST_CONFIG_FILE^^} +USE_CUSTOM_POST_CONFIG_FILE=$(echo_uppercase $USE_CUSTOM_POST_CONFIG_FILE) if [ "$USE_CUSTOM_POST_CONFIG_FILE" = "TRUE" ] || \ [ "$USE_CUSTOM_POST_CONFIG_FILE" = "YES" ]; then USE_CUSTOM_POST_CONFIG_FILE="TRUE" @@ -1224,7 +1251,7 @@ if [ "${SUB_HOURLY_POST}" = "TRUE" ]; then # digits. # mnts_or_null=$( printf "%s" "${DT_SUBHOURLY_POST_MNTS}" | \ - sed -n -r -e "s/^([0-9])([0-9])?$/\1\2/p" ) + $SED -n -r -e "s/^([0-9])([0-9])?$/\1\2/p" ) if [ -z "${mnts_or_null}" ]; then print_err_msg_exit "\ When performing sub-hourly post (i.e. SUB_HOURLY_POST set to \"TRUE\"), @@ -1318,7 +1345,7 @@ fi if [ "${EXPT_BASEDIR:0:1}" != "/" ]; then EXPT_BASEDIR="${SR_WX_APP_TOP_DIR}/../expt_dirs/${EXPT_BASEDIR}" fi -EXPT_BASEDIR="$( readlink -m ${EXPT_BASEDIR} )" +EXPT_BASEDIR="$( $READLINK -m ${EXPT_BASEDIR} )" mkdir_vrfy -p "${EXPT_BASEDIR}" # #----------------------------------------------------------------------- @@ -1577,7 +1604,7 @@ check_var_valid_value "USE_USER_STAGED_EXTRN_FILES" "valid_vals_USE_USER_STAGED_ # Set USE_USER_STAGED_EXTRN_FILES to either "TRUE" or "FALSE" so we don't # have to consider other valid values later on. # -USE_USER_STAGED_EXTRN_FILES=${USE_USER_STAGED_EXTRN_FILES^^} +USE_USER_STAGED_EXTRN_FILES=$(echo_uppercase $USE_USER_STAGED_EXTRN_FILES) if [ "${USE_USER_STAGED_EXTRN_FILES}" = "YES" ]; then USE_USER_STAGED_EXTRN_FILES="TRUE" elif [ "${USE_USER_STAGED_EXTRN_FILES}" = "NO" ]; then @@ -1624,7 +1651,7 @@ check_var_valid_value "DO_ENSEMBLE" "valid_vals_DO_ENSEMBLE" # Set DO_ENSEMBLE to either "TRUE" or "FALSE" so we don't have to consider # other valid values later on. # -DO_ENSEMBLE=${DO_ENSEMBLE^^} +DO_ENSEMBLE=$(echo_uppercase $DO_ENSEMBLE) if [ "$DO_ENSEMBLE" = "TRUE" ] || \ [ "$DO_ENSEMBLE" = "YES" ]; then DO_ENSEMBLE="TRUE" @@ -2125,12 +2152,14 @@ fi #----------------------------------------------------------------------- # # Create a new experiment directory. Note that at this point we are -# guaranteed that there is no preexisting experiment directory. +# guaranteed that there is no preexisting experiment directory. For +# platforms with no workflow manager, we need to create LOGDIR as well, +# since it won't be created later at runtime. # #----------------------------------------------------------------------- # mkdir_vrfy -p "$EXPTDIR" - +mkdir_vrfy -p "$LOGDIR" # #----------------------------------------------------------------------- # @@ -2257,7 +2286,7 @@ check_var_valid_value "WRITE_DOPOST" "valid_vals_WRITE_DOPOST" # Set WRITE_DOPOST to either "TRUE" or "FALSE" so we don't have to consider # other valid values later on. # -WRITE_DOPOST=${WRITE_DOPOST^^} +WRITE_DOPOST=$(echo_uppercase $WRITE_DOPOST) if [ "$WRITE_DOPOST" = "TRUE" ] || \ [ "$WRITE_DOPOST" = "YES" ]; then WRITE_DOPOST="TRUE" @@ -2287,7 +2316,7 @@ check_var_valid_value "QUILTING" "valid_vals_QUILTING" # Set QUILTING to either "TRUE" or "FALSE" so we don't have to consider # other valid values later on. # -QUILTING=${QUILTING^^} +QUILTING=$(echo_uppercase $QUILTING) if [ "$QUILTING" = "TRUE" ] || \ [ "$QUILTING" = "YES" ]; then QUILTING="TRUE" @@ -2307,7 +2336,7 @@ check_var_valid_value "PRINT_ESMF" "valid_vals_PRINT_ESMF" # Set PRINT_ESMF to either "TRUE" or "FALSE" so we don't have to consider # other valid values later on. # -PRINT_ESMF=${PRINT_ESMF^^} +PRINT_ESMF=$(echo_uppercase $PRINT_ESMF) if [ "${PRINT_ESMF}" = "TRUE" ] || \ [ "${PRINT_ESMF}" = "YES" ]; then PRINT_ESMF="TRUE" @@ -2458,7 +2487,7 @@ cp_vrfy $USHDIR/${EXPT_DEFAULT_CONFIG_FN} ${GLOBAL_VAR_DEFNS_FP} # # Read all lines of GLOBAL_VAR_DEFNS file into the variable line_list. -line_list=$( sed -r -e "s/(.*)/\1/g" ${GLOBAL_VAR_DEFNS_FP} ) +line_list=$( $SED -r -e "s/(.*)/\1/g" ${GLOBAL_VAR_DEFNS_FP} ) # # Loop through the lines in line_list and concatenate lines ending with # the line bash continuation character "\". @@ -2499,7 +2528,7 @@ done <<< "${line_list}" #----------------------------------------------------------------------- # # Also should remove trailing whitespace... -line_list=$( sed -r \ +line_list=$( $SED -r \ -e "s/^([ ]*)([^ ]+.*)/\2/g" \ -e "/^#.*/d" \ -e "/^$/d" \ @@ -2545,7 +2574,7 @@ str_to_insert=${str_to_insert//$'\n'/\\n} # the string "#!", e.g. "#!/bin/bash"). # regexp="(^#!.*)" -sed -i -r -e "s|$regexp|\1\n\n${str_to_insert}\n|g" ${GLOBAL_VAR_DEFNS_FP} +$SED -i -r -e "s|$regexp|\1\n\n${str_to_insert}\n|g" ${GLOBAL_VAR_DEFNS_FP} # # Loop through the lines in line_list. # @@ -2559,7 +2588,7 @@ while read crnt_line; do # or more characters representing the value that the variable is being # set to. # - var_name=$( printf "%s" "${crnt_line}" | sed -n -r -e "s/^([^ ]*)=.*/\1/p" ) + var_name=$( printf "%s" "${crnt_line}" | $SED -n -r -e "s/^([^ ]*)=.*/\1/p" ) #echo #echo "============================" #printf "%s\n" "var_name = \"${var_name}\"" @@ -2766,7 +2795,7 @@ FV3_NML_ENSMEM_FPS=( $( printf "\"%s\" " "${FV3_NML_ENSMEM_FPS[@]}" )) GLOBAL_VAR_DEFNS_FP="${GLOBAL_VAR_DEFNS_FP}" # Try this at some point instead of hard-coding it as above; it's a more # flexible approach (if it works). -#GLOBAL_VAR_DEFNS_FP=$( readlink -f "${BASH_SOURCE[0]}" ) +#GLOBAL_VAR_DEFNS_FP=$( $READLINK -f "${BASH_SOURCE[0]}" ) DATA_TABLE_TMPL_FN="${DATA_TABLE_TMPL_FN}" DIAG_TABLE_TMPL_FN="${DIAG_TABLE_TMPL_FN}" @@ -3011,6 +3040,7 @@ FVCOM_FILE="${FVCOM_FILE}" # NCORES_PER_NODE="${NCORES_PER_NODE}" PE_MEMBER01="${PE_MEMBER01}" +RUN_CMD_FCST="${RUN_CMD_FCST}" # #----------------------------------------------------------------------- # diff --git a/ush/source_util_funcs.sh b/ush/source_util_funcs.sh index bc6a2f083..375543d35 100644 --- a/ush/source_util_funcs.sh +++ b/ush/source_util_funcs.sh @@ -8,7 +8,11 @@ function source_util_funcs() { # #----------------------------------------------------------------------- # - local scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) + if [[ $(uname -s) == Darwin ]]; then + local scrfunc_fp=$( greadlink -f "${BASH_SOURCE[0]}" ) + else + local scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) + fi local scrfunc_fn=$( basename "${scrfunc_fp}" ) local scrfunc_dir=$( dirname "${scrfunc_fp}" ) # @@ -48,6 +52,15 @@ function source_util_funcs() { # #----------------------------------------------------------------------- # +# Source the file that defines MacOS-specific UNIX command-line +# utilities, that mimic the functionality of the GNU equivalents +# +#----------------------------------------------------------------------- +# + . ${bashutils_dir}/define_macos_utilities.sh +# +#----------------------------------------------------------------------- +# # Source the file containing the functions that print out messages. # #----------------------------------------------------------------------- @@ -82,6 +95,15 @@ function source_util_funcs() { # #----------------------------------------------------------------------- # +# Source the file containing the functions that will echo given strings +# as uppercase or lowercase +# +#----------------------------------------------------------------------- +# + . ${bashutils_dir}/change_case.sh +# +#----------------------------------------------------------------------- +# # Source the file containing the function that checks for preexisting # directories or files and handles them according to a specified method # (which can be one of "delete", "rename", and "quit"). diff --git a/ush/valid_param_vals.sh b/ush/valid_param_vals.sh index d5eb39b9b..f42093fa6 100644 --- a/ush/valid_param_vals.sh +++ b/ush/valid_param_vals.sh @@ -3,9 +3,10 @@ # valid_vals_RUN_ENVIR=("nco" "community") valid_vals_VERBOSE=("TRUE" "true" "YES" "yes" "FALSE" "false" "NO" "no") -valid_vals_MACHINE=("WCOSS_CRAY" "WCOSS_DELL_P3" "HERA" "ORION" "JET" "ODIN" "CHEYENNE" "STAMPEDE") +valid_vals_MACHINE=("WCOSS_CRAY" "WCOSS_DELL_P3" "HERA" "ORION" "JET" "ODIN" "CHEYENNE" "STAMPEDE" "LINUX" "MACOS") valid_vals_SCHED=("slurm" "pbspro" "lsf" "lsfcray" "none") valid_vals_FCST_MODEL=("ufs-weather-model" "fv3gfs_aqm") +valid_vals_WORKFLOW_MANAGER=("rocoto" "none") valid_vals_PREDEF_GRID_NAME=( \ "RRFS_CONUS_25km" \ "RRFS_CONUS_13km" \ From ca8eea3d438befa45fdff79e69db6d727d918d4e Mon Sep 17 00:00:00 2001 From: "Chan-Hoo.Jeon-NOAA" <60152248+chan-hoo@users.noreply.github.com> Date: Thu, 23 Sep 2021 06:18:36 -0400 Subject: [PATCH 03/16] Add option for fv3gfs_aqm to run_post (#585) * Add option for fv3gfs_aqm * Change parameter name --- scripts/exregional_run_post.sh | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/scripts/exregional_run_post.sh b/scripts/exregional_run_post.sh index b944ea169..9b0f26a42 100755 --- a/scripts/exregional_run_post.sh +++ b/scripts/exregional_run_post.sh @@ -182,7 +182,11 @@ to the temporary work directory (tmp_dir): tmp_dir = \"${tmp_dir}\" ====================================================================" else - post_config_fp="${UPP_DIR}/parm/postxconfig-NT-fv3lam.txt" + if [ ${FCST_MODEL} = "fv3gfs_aqm" ]; then + post_config_fp="${UPP_DIR}/parm/postxconfig-NT-fv3lam_cmaq.txt" + else + post_config_fp="${UPP_DIR}/parm/postxconfig-NT-fv3lam.txt" + fi print_info_msg " ==================================================================== Copying the default post flat file specified by post_config_fp to the @@ -260,6 +264,11 @@ post_mn=${post_time:10:2} # # Create the input text file to the post-processor executable. # +if [ ${FCST_MODEL} = "fv3gfs_aqm" ]; then + post_itag_add="aqfcmaq_on=.true.," +else + post_itag_add="" +fi cat > itag < Date: Tue, 28 Sep 2021 08:41:17 -0600 Subject: [PATCH 04/16] Remove Julie from CODEOWNERS (#609) --- .github/CODEOWNERS | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 40196de74..c05ae7079 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -3,7 +3,7 @@ # These owners will be the default owners for everything in the repo. #* @defunkt -* @JulieSchramm @mkavulich @gsketefian @JeffBeck-NOAA @RatkoVasic-NOAA @BenjaminBlake-NOAA +* @mkavulich @gsketefian @JeffBeck-NOAA @RatkoVasic-NOAA @BenjaminBlake-NOAA # Order is important. The last matching pattern has the most precedence. # So if a pull request only touches javascript files, only these owners From dff4c7fbeace32c9ec0ebb605f2975cfea1ecc60 Mon Sep 17 00:00:00 2001 From: JeffBeck-NOAA <55201531+JeffBeck-NOAA@users.noreply.github.com> Date: Wed, 6 Oct 2021 13:16:51 -0600 Subject: [PATCH 05/16] Add do_gsl* namelist entries to YAML file for FV3_HRRR SDF. (#597) --- ush/templates/FV3.input.yml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/ush/templates/FV3.input.yml b/ush/templates/FV3.input.yml index 4d7c99b54..8e9b43fe5 100644 --- a/ush/templates/FV3.input.yml +++ b/ush/templates/FV3.input.yml @@ -70,6 +70,9 @@ FV3_HRRR: do_mynnsfclay: True do_sfcperts: !!python/none gwd_opt: 3 + do_gsl_drag_ss: True + do_gsl_drag_tofd: True + do_gsl_drag_ls_bl: True iaer: 5111 icliq_sw: 2 imfdeepcnv: -1 From 73aadeb970068dc97f7351637b77313db8731013 Mon Sep 17 00:00:00 2001 From: "Chan-Hoo.Jeon-NOAA" <60152248+chan-hoo@users.noreply.github.com> Date: Thu, 7 Oct 2021 06:22:14 -0400 Subject: [PATCH 06/16] Fix unbound SED for cron commands in the launch script (#606) * Add source_util_funcs to launch script --- ush/launch_FV3LAM_wflow.sh | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/ush/launch_FV3LAM_wflow.sh b/ush/launch_FV3LAM_wflow.sh index 72cdd74c7..fb2c71c96 100755 --- a/ush/launch_FV3LAM_wflow.sh +++ b/ush/launch_FV3LAM_wflow.sh @@ -25,6 +25,10 @@ else fi scrfunc_fn=$( basename "${scrfunc_fp}" ) scrfunc_dir=$( dirname "${scrfunc_fp}" ) + +ushdir="${scrfunc_dir}" +. $ushdir/source_util_funcs.sh + # #----------------------------------------------------------------------- # @@ -309,8 +313,8 @@ while read -r line; do # if [ $i -gt 0 ]; then im1=$((i-1)) - cycle_str[im1]=$( echo "$line" | sed -r -n -e "s/${regex_search}/\1/p" ) - cycle_status[im1]=$( echo "$line" | sed -r -n -e "s/${regex_search}/\2/p" ) + cycle_str[im1]=$( echo "$line" | $SED -r -n -e "s/${regex_search}/\1/p" ) + cycle_status[im1]=$( echo "$line" | $SED -r -n -e "s/${regex_search}/\2/p" ) fi i=$((i+1)) done <<< "${rocotostat_output}" From bdca712e09875871a83f7708c243166dba36ed3c Mon Sep 17 00:00:00 2001 From: Christina Holt <56881914+christinaholtNOAA@users.noreply.github.com> Date: Tue, 19 Oct 2021 17:12:11 -0600 Subject: [PATCH 07/16] Adding an environment file for installing pygraf. (#604) --- ush/Python/environment.yml | 15 +++++++++++++++ 1 file changed, 15 insertions(+) create mode 100644 ush/Python/environment.yml diff --git a/ush/Python/environment.yml b/ush/Python/environment.yml new file mode 100644 index 000000000..8e08987ce --- /dev/null +++ b/ush/Python/environment.yml @@ -0,0 +1,15 @@ +name: pygraf +channels: + - conda-forge + - defaults +dependencies: + - python=3.7* + - basemap=1.2* + - basemap-data-hires=1.2.1 + - pynio=1.5.5 + - matplotlib=3.2* + - metpy=0.12.1 + - pylint=2.4* + - pytest=6.1* + - pyyaml=5.3.1 + - xarray=0.15.1 From 7da0c46750b30cd7eeba35e03448f6a9a9f97e42 Mon Sep 17 00:00:00 2001 From: "Chan-Hoo.Jeon-NOAA" <60152248+chan-hoo@users.noreply.github.com> Date: Fri, 22 Oct 2021 07:56:40 -0400 Subject: [PATCH 08/16] Modify workflow environment for orion (#616) --- modulefiles/tasks/orion/make_grid.local | 3 ++- modulefiles/tasks/orion/make_ics.local | 2 +- modulefiles/tasks/orion/make_lbcs.local | 2 +- modulefiles/tasks/orion/run_fcst.local | 2 +- 4 files changed, 5 insertions(+), 4 deletions(-) diff --git a/modulefiles/tasks/orion/make_grid.local b/modulefiles/tasks/orion/make_grid.local index 93a76f6ca..4de2a79ca 100644 --- a/modulefiles/tasks/orion/make_grid.local +++ b/modulefiles/tasks/orion/make_grid.local @@ -1,5 +1,6 @@ #%Module + module use -a /apps/contrib/miniconda3-noaa-gsl/modulefiles -module load miniconda3 +module load miniconda3/3.8 setenv SRW_ENV regional_workflow diff --git a/modulefiles/tasks/orion/make_ics.local b/modulefiles/tasks/orion/make_ics.local index 93a76f6ca..3703a9ba1 100644 --- a/modulefiles/tasks/orion/make_ics.local +++ b/modulefiles/tasks/orion/make_ics.local @@ -1,5 +1,5 @@ #%Module module use -a /apps/contrib/miniconda3-noaa-gsl/modulefiles -module load miniconda3 +module load miniconda3/3.8 setenv SRW_ENV regional_workflow diff --git a/modulefiles/tasks/orion/make_lbcs.local b/modulefiles/tasks/orion/make_lbcs.local index 93a76f6ca..3703a9ba1 100644 --- a/modulefiles/tasks/orion/make_lbcs.local +++ b/modulefiles/tasks/orion/make_lbcs.local @@ -1,5 +1,5 @@ #%Module module use -a /apps/contrib/miniconda3-noaa-gsl/modulefiles -module load miniconda3 +module load miniconda3/3.8 setenv SRW_ENV regional_workflow diff --git a/modulefiles/tasks/orion/run_fcst.local b/modulefiles/tasks/orion/run_fcst.local index 93a76f6ca..3703a9ba1 100644 --- a/modulefiles/tasks/orion/run_fcst.local +++ b/modulefiles/tasks/orion/run_fcst.local @@ -1,5 +1,5 @@ #%Module module use -a /apps/contrib/miniconda3-noaa-gsl/modulefiles -module load miniconda3 +module load miniconda3/3.8 setenv SRW_ENV regional_workflow From 43e8915cfa1338e1ddea21a0302232c9e30a74af Mon Sep 17 00:00:00 2001 From: "Chan-Hoo.Jeon-NOAA" <60152248+chan-hoo@users.noreply.github.com> Date: Wed, 27 Oct 2021 06:34:20 -0400 Subject: [PATCH 09/16] increase wtime_make_orog for RRFS NA 3km on WCOSS (#622) * increase wtime_make_orog for RRFS_NA_3km on wcoss * remove ppn_run_fcst from WE2E script for RRFS_NA_3km * Add EMC configuration in comment --- ...d_RRFS_NA_3km_ics_FV3GFS_lbcs_FV3GFS_suite_RRFS_v1alpha.sh | 2 -- ush/config_defaults.sh | 2 +- ush/set_predef_grid_params.sh | 4 ++-- 3 files changed, 3 insertions(+), 5 deletions(-) diff --git a/tests/WE2E/test_configs/grids_extrn_mdls_suites_community/config.grid_RRFS_NA_3km_ics_FV3GFS_lbcs_FV3GFS_suite_RRFS_v1alpha.sh b/tests/WE2E/test_configs/grids_extrn_mdls_suites_community/config.grid_RRFS_NA_3km_ics_FV3GFS_lbcs_FV3GFS_suite_RRFS_v1alpha.sh index 9b4e0da93..a01a30807 100644 --- a/tests/WE2E/test_configs/grids_extrn_mdls_suites_community/config.grid_RRFS_NA_3km_ics_FV3GFS_lbcs_FV3GFS_suite_RRFS_v1alpha.sh +++ b/tests/WE2E/test_configs/grids_extrn_mdls_suites_community/config.grid_RRFS_NA_3km_ics_FV3GFS_lbcs_FV3GFS_suite_RRFS_v1alpha.sh @@ -45,8 +45,6 @@ PPN_MAKE_ICS="4" PPN_MAKE_LBCS="4" WTIME_MAKE_LBCS="01:00:00" -PPN_RUN_FCST="24" - NNODES_RUN_POST="6" PPN_RUN_POST="12" diff --git a/ush/config_defaults.sh b/ush/config_defaults.sh index dd6a2968c..35c744039 100644 --- a/ush/config_defaults.sh +++ b/ush/config_defaults.sh @@ -1515,7 +1515,7 @@ PPN_VX_ENSPOINT_PROB="1" # Walltimes. # WTIME_MAKE_GRID="00:20:00" -WTIME_MAKE_OROG="00:20:00" +WTIME_MAKE_OROG="01:00:00" WTIME_MAKE_SFC_CLIMO="00:20:00" WTIME_GET_EXTRN_ICS="00:45:00" WTIME_GET_EXTRN_LBCS="00:45:00" diff --git a/ush/set_predef_grid_params.sh b/ush/set_predef_grid_params.sh index aca2a26e3..dd664670a 100644 --- a/ush/set_predef_grid_params.sh +++ b/ush/set_predef_grid_params.sh @@ -1227,8 +1227,8 @@ case ${PREDEF_GRID_NAME} in DT_ATMOS="${DT_ATMOS:-36}" - LAYOUT_X="${LAYOUT_X:-18}" - LAYOUT_Y="${LAYOUT_Y:-36}" + LAYOUT_X="${LAYOUT_X:-18}" # 40 - EMC operational configuration + LAYOUT_Y="${LAYOUT_Y:-36}" # 45 - EMC operational configuration BLOCKSIZE="${BLOCKSIZE:-28}" if [ "$QUILTING" = "TRUE" ]; then From e47127471f61019e6b021892ce0c025176ec9a7c Mon Sep 17 00:00:00 2001 From: David Wright Date: Wed, 27 Oct 2021 18:12:16 -0400 Subject: [PATCH 10/16] Allow FVCOM to be used in Warm and Cold Starts (#620) * Change fvcom_to_FV3 use to allow for warm/cold start sfc data and time selection in a FVCOM file * Update valid values for FVCOM_WCSTART --- scripts/exregional_make_ics.sh | 6 +++++- ush/config_defaults.sh | 7 +++++++ ush/setup.sh | 4 ++++ ush/valid_param_vals.sh | 1 + 4 files changed, 17 insertions(+), 1 deletion(-) diff --git a/scripts/exregional_make_ics.sh b/scripts/exregional_make_ics.sh index ec0fee623..951477c93 100755 --- a/scripts/exregional_make_ics.sh +++ b/scripts/exregional_make_ics.sh @@ -658,8 +658,10 @@ mv_vrfy gfs.bndy.nc ${ics_dir}/gfs_bndy.tile${TILE_RGNL}.000.nc # if [ "${USE_FVCOM}" = "TRUE" ]; then +#Format for fvcom_time: YYYY-MM-DDTHH:00:00.000000 fvcom_exec_fn="fvcom_to_FV3" fvcom_exec_fp="$EXECDIR/${fvcom_exec_fn}" + fvcom_time="${DATE_FIRST_CYCL:0:4}-${DATE_FIRST_CYCL:4:2}-${DATE_FIRST_CYCL:6:2}T${CYCL_HRS[0]}:00:00.000000" if [ ! -f "${fvcom_exec_fp}" ]; then print_err_msg_exit "\ The executable (fvcom_exec_fp) for processing FVCOM data onto FV3-LAM @@ -680,13 +682,15 @@ Please check the following user defined variables: cp_vrfy ${fvcom_data_fp} ${ics_dir}/fvcom.nc cd_vrfy ${ics_dir} - ${APRUN} ${fvcom_exec_fn} sfc_data.tile${TILE_RGNL}.halo${NH0}.nc fvcom.nc || \ + ${APRUN} ${fvcom_exec_fn} sfc_data.tile${TILE_RGNL}.halo${NH0}.nc fvcom.nc ${FVCOM_WCSTART} ${fvcom_time}|| \ print_err_msg_exit "\ Call to executable (fvcom_exe) to modify sfc fields for FV3-LAM failed: fvcom_exe = \"${fvcom_exe}\" The following variables were being used: FVCOM_DIR = \"${FVCOM_DIR}\" FVCOM_FILE = \"${FVCOM_FILE}\" + fvcom_time = \"${fvcom_time}\" + FVCOM_WCSTART = \"${FVCOM_WCSTART}\" ics_dir = \"${ics_dir}\" fvcom_exe_dir = \"${fvcom_exe_dir}\" fvcom_exe = \"${fvcom_exe}\"" diff --git a/ush/config_defaults.sh b/ush/config_defaults.sh index 35c744039..1ce86b9b1 100644 --- a/ush/config_defaults.sh +++ b/ush/config_defaults.sh @@ -1723,6 +1723,12 @@ HALO_BLEND="10" # FV3-LAM grid. This flag will be used in make_ics to modify sfc_data.nc # after chgres_cube is run by running the routine process_FVCOM.exe # +# FVCOM_WCSTART: +# Define if this is a "warm" start or a "cold" start. Setting this to +# "warm" will read in sfc_data.nc generated in a RESTART directory. +# Setting this to "cold" will read in the sfc_data.nc generated from +# chgres_cube in the make_ics portion of the workflow. +# # FVCOM_DIR: # User defined directory where FVCOM data already interpolated to FV3-LAM # grid is located. File name in this path should be "fvcom.nc" to allow @@ -1735,6 +1741,7 @@ HALO_BLEND="10" #------------------------------------------------------------------------ # USE_FVCOM="FALSE" +FVCOM_WCSTART="cold" FVCOM_DIR="/user/defined/dir/to/fvcom/data" FVCOM_FILE="fvcom.nc" # diff --git a/ush/setup.sh b/ush/setup.sh index 96b1ef071..a166fafcc 100755 --- a/ush/setup.sh +++ b/ush/setup.sh @@ -450,10 +450,13 @@ fi # # Make sure that USE_FVCOM is set to a valid value and assign directory # and file names. +# +# Make sure that FVCOM_WCSTART is set to lowercase "warm" or "cold" # #----------------------------------------------------------------------- # check_var_valid_value "USE_FVCOM" "valid_vals_USE_FVCOM" +check_var_valid_value "FVCOM_WCSTART" "valid_vals_FVCOM_WCSTART" # # Set USE_FVCOM to either "TRUE" or "FALSE" so we don't have to consider # other valid values later on. @@ -466,6 +469,7 @@ elif [ "$USE_FVCOM" = "FALSE" ] || \ [ "$USE_FVCOM" = "NO" ]; then USE_FVCOM="FALSE" fi +FVCOM_WCSTART=$(echo_lowercase $FVCOM_WCSTART) # #----------------------------------------------------------------------- # diff --git a/ush/valid_param_vals.sh b/ush/valid_param_vals.sh index f42093fa6..1b3acf00a 100644 --- a/ush/valid_param_vals.sh +++ b/ush/valid_param_vals.sh @@ -72,6 +72,7 @@ valid_vals_DO_SPP=("TRUE" "true" "YES" "yes" "FALSE" "false" "NO" "no") valid_vals_DO_SKEB=("TRUE" "true" "YES" "yes" "FALSE" "false" "NO" "no") valid_vals_USE_ZMTNBLCK=("TRUE" "true" "YES" "yes" "FALSE" "false" "NO" "no") valid_vals_USE_FVCOM=("TRUE" "true" "YES" "yes" "FALSE" "false" "NO" "no") +valid_vals_FVCOM_WCSTART=("warm" "WARM" "cold" "COLD") valid_vals_COMPILER=("intel" "gnu") valid_vals_SUB_HOURLY_POST=("TRUE" "true" "YES" "yes" "FALSE" "false" "NO" "no") valid_vals_DT_SUBHOURLY_POST_MNTS=("1" "01" "2" "02" "3" "03" "4" "04" "5" "05" "6" "06" "10" "12" "15" "20" "30") From 5fcddb4d8cdac9b134d83df8176e8f20dd0d28f0 Mon Sep 17 00:00:00 2001 From: Christina Holt <56881914+christinaholtNOAA@users.noreply.github.com> Date: Wed, 10 Nov 2021 17:12:02 -0700 Subject: [PATCH 11/16] Add support for Rocoto with generic LINUX platform (#617) * Adding all changes. Untested at this point. * Chris's additions for running on AWS. * Changes needed for AWS platform. * Clean up comment to match code. * Fix spelling error. * More docs for choosing linux + rocoto. * Require NCORES_PER_NODE for all machines. * Ensure variables are expanded within RUN_CMD --- scripts/exregional_make_ics.sh | 1 + scripts/exregional_make_lbcs.sh | 1 + scripts/exregional_make_orog.sh | 2 + scripts/exregional_run_fcst.sh | 2 + ush/config_defaults.sh | 33 +++++++++- ush/launch_FV3LAM_wflow.sh | 54 +++-------------- ush/load_modules_run_task.sh | 104 ++++++++++---------------------- ush/setup.sh | 58 ++++++++++++------ ush/templates/FV3LAM_wflow.xml | 2 +- 9 files changed, 119 insertions(+), 138 deletions(-) diff --git a/scripts/exregional_make_ics.sh b/scripts/exregional_make_ics.sh index 951477c93..dd64a4999 100755 --- a/scripts/exregional_make_ics.sh +++ b/scripts/exregional_make_ics.sh @@ -131,6 +131,7 @@ case "$MACHINE" in ;; "LINUX") + ulimit -s unlimited APRUN=$RUN_CMD_UTILS ;; diff --git a/scripts/exregional_make_lbcs.sh b/scripts/exregional_make_lbcs.sh index 1b1f84ff8..01aeaed96 100755 --- a/scripts/exregional_make_lbcs.sh +++ b/scripts/exregional_make_lbcs.sh @@ -131,6 +131,7 @@ case "$MACHINE" in ;; "LINUX") + ulimit -s unlimited APRUN=$RUN_CMD_UTILS ;; diff --git a/scripts/exregional_make_orog.sh b/scripts/exregional_make_orog.sh index e90b3b762..afd40d74d 100755 --- a/scripts/exregional_make_orog.sh +++ b/scripts/exregional_make_orog.sh @@ -153,6 +153,8 @@ case "$MACHINE" in "LINUX") APRUN=time + ulimit -s unlimited + ulimit -a ;; *) diff --git a/scripts/exregional_run_fcst.sh b/scripts/exregional_run_fcst.sh index 4b16f79d9..aae18a749 100755 --- a/scripts/exregional_run_fcst.sh +++ b/scripts/exregional_run_fcst.sh @@ -159,6 +159,8 @@ case "$MACHINE" in ;; "LINUX") + ulimit -s unlimited + ulimit -a APRUN=$RUN_CMD_FCST ;; diff --git a/ush/config_defaults.sh b/ush/config_defaults.sh index 1ce86b9b1..ba3711f90 100644 --- a/ush/config_defaults.sh +++ b/ush/config_defaults.sh @@ -41,7 +41,12 @@ RUN_ENVIR="nco" # Set machine and queue parameters. Definitions: # # MACHINE: -# Machine on which the workflow will run. +# Machine on which the workflow will run. If you are NOT on a named, +# supported platform, and you want to use the Rocoto workflow manager, +# you will need set MACHINE="linux" and WORKFLOW_MANAGER="rocoto". This +# combination will assume a Slurm batch manager when generating the XML. +# Please see ush/valid_param_vals.sh for a full list of supported +# platforms. # # ACCOUNT: # The account under which to submit jobs to the queue. @@ -49,7 +54,27 @@ RUN_ENVIR="nco" # WORKFLOW_MANAGER: # The workflow manager to use (e.g. rocoto). This is set to "none" by # default, but if the machine name is set to a platform that supports -# rocoto, this will be overwritten and set to "rocoto". +# rocoto, this will be overwritten and set to "rocoto". If set +# explicitly to rocoto along with the use of the MACHINE=linux target, +# the configuration layer assumes a Slurm batch manager when generating +# the XML. Valid options: "rocoto" or "none" +# +# NCORES_PER_NODE: +# The number of cores available per node on the compute platform. Set +# for supported platforms in setup.sh, but is now also configurable for +# all platforms. +# +# LMOD_PATH: +# Path to the LMOD sh file on your Linux system. Is set automatically +# for supported machines. +# +# BUILD_ENV_FN: +# Name of alternative build environment file to use if using an +# unsupported platform. Is set automatically for supported machines. +# +# WFLOW_ENV_FN: +# Name of alternative workflow environment file to use if using an +# unsupported platform. Is set automatically for supported machines. # # SCHED: # The job scheduler to use (e.g. slurm). Set this to an empty string in @@ -109,6 +134,10 @@ RUN_ENVIR="nco" MACHINE="BIG_COMPUTER" ACCOUNT="project_name" WORKFLOW_MANAGER="none" +NCORES_PER_NODE="" +LMOD_PATH="" +BUILD_ENV_FN="" +WFLOW_ENV_FN="" SCHED="" PARTITION_DEFAULT="" QUEUE_DEFAULT="" diff --git a/ush/launch_FV3LAM_wflow.sh b/ush/launch_FV3LAM_wflow.sh index fb2c71c96..fe7ec8df6 100755 --- a/ush/launch_FV3LAM_wflow.sh +++ b/ush/launch_FV3LAM_wflow.sh @@ -88,6 +88,7 @@ fi #----------------------------------------------------------------------- # . $exptdir/var_defns.sh +. ${USHDIR}/source_util_funcs.sh # #----------------------------------------------------------------------- # @@ -105,13 +106,7 @@ expt_name="${EXPT_SUBDIR}" # #----------------------------------------------------------------------- # -if [ "$MACHINE" = "CHEYENNE" ]; then - module use -a /glade/p/ral/jntp/UFS_SRW_app/modules/ - module load rocoto -elif [ "$MACHINE" = "ORION" ]; then - module purge - module load contrib rocoto -elif [ "$MACHINE" = "WCOSS_DELL_P3" ]; then +if [ "$MACHINE" = "WCOSS_DELL_P3" ]; then module purge module load lsf/10.1 module use /gpfs/dell3/usrx/local/dev/emc_rocoto/modulefiles/ @@ -122,8 +117,14 @@ elif [ "$MACHINE" = "WCOSS_CRAY" ]; then module use -a /usrx/local/emc_rocoto/modulefiles module load rocoto/1.3.0rc2 else + machine=$(echo_lowercase $MACHINE) + env_fn=${WFLOW_ENV_FN:-"wflow_${machine}.env"} + env_fp="${SR_WX_APP_TOP_DIR}/env/${env_fn}" module purge - module load rocoto + source "${env_fp}" || print_err_msg_exit "\ + Sourcing platform-specific environment file (env_fp) for + the workflow task failed : + env_fp = \"${env_fp}\"" fi # #----------------------------------------------------------------------- @@ -166,35 +167,7 @@ cd "$exptdir" #----------------------------------------------------------------------- # -#rocotorun_output=$( ls -alF ) -#echo -#echo "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA" -#echo "${rocotorun_output}" -#echo "BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB" - -#rocotorun_output=$( \ -#rocotorun -w "${WFLOW_XML_FN}" -d "${rocoto_database_fn}" -v 10 \ -#) -#rocotorun_output=$( (rocotorun -w "${WFLOW_XML_FN}" -d "${rocoto_database_fn}" -v 10) 2>&1 ) # This freezes the script. -#rocotorun_output=$( (rocotorun -w "${WFLOW_XML_FN}" -d "${rocoto_database_fn}" -v 10) 1>&2 ) # This leaves rocotorun_output empty. -#rocotorun_output=$( rocotorun -w "${WFLOW_XML_FN}" -d "${rocoto_database_fn}" -v 10 ) -#{ error=$(command 2>&1 1>&$out); } {out}>&1 -#{ rocotorun_output=$( rocotorun -w "${WFLOW_XML_FN}" -d "${rocoto_database_fn}" -v 10 2>&1 1>&$out); } {out}>&1 # This freezes the script. - -# -# Ideally, the following two lines should work, but for some reason the -# output of rocotorun cannot be captured in a variable using the $(...) -# notation. Maybe it's not being written to stdout, although I tried -# redirecting stderr to stdout and other tricks but nothing seemed to -# work. For this reason, below we first redirect the output of rocoto- -# run to a temporary file and then read in the contents of that file in- -# to the rocotorun_output variable using the cat command. -# -#rocotorun_cmd="rocotorun -w \"${WFLOW_XML_FN}\" -d \"${rocoto_database_fn}\" -v 10" -#rocotorun_output=$( eval ${rocotorun_cmd} 2>&1 ) -# tmp_fn="rocotorun_output.txt" -#rocotorun_cmd="rocotorun -w \"${WFLOW_XML_FN}\" -d \"${rocoto_database_fn}\" -v 10 > ${tmp_fn}" rocotorun_cmd="rocotorun -w \"${WFLOW_XML_FN}\" -d \"${rocoto_database_fn}\" -v 10" eval ${rocotorun_cmd} > ${tmp_fn} 2>&1 rocotorun_output=$( cat "${tmp_fn}" ) @@ -221,18 +194,9 @@ done <<< "${rocotorun_output}" # #----------------------------------------------------------------------- # -#rocotostat_cmd="{ pwd; rocotostat -w \"${WFLOW_XML_FN}\" -d \"${rocoto_database_fn}\" -v 10; }" -#rocotostat_cmd="{ pwd; ls -alF; rocotostat -w ${WFLOW_XML_FN} -d ${rocoto_database_fn} -v 10; }" -#rocotostat_cmd="{ pwd; ls -alF; rocotostat -w \"${WFLOW_XML_FN}\" -d \"${rocoto_database_fn}\" -v 10; }" -#rocotostat_cmd="{ pwd; rocotostat -w \"${WFLOW_XML_FN}\" -d \"${rocoto_database_fn}\" -v 10; }" -#rocotostat_cmd="{ rocotostat -w \"${WFLOW_XML_FN}\" -d \"${rocoto_database_fn}\" -v 10; }" rocotostat_cmd="rocotostat -w \"${WFLOW_XML_FN}\" -d \"${rocoto_database_fn}\" -v 10" -#rocotostat_output=$( pwd; rocotostat -w "${WFLOW_XML_FN}" -d "${rocoto_database_fn}" -v 10 2>&1 ) -#rocotostat_output=$( rocotostat -w "${WFLOW_XML_FN}" -d "${rocoto_database_fn}" -v 10 2>&1 ) rocotostat_output=$( eval ${rocotostat_cmd} 2>&1 ) -#rocotostat_output=$( ${rocotostat_cmd} 2>&1 ) -#rocotostat_output=$( { pwd; ls -alF; } 2>&1 ) error_msg="DEAD" while read -r line; do grep_output=$( printf "$line" | grep "${error_msg}" ) diff --git a/ush/load_modules_run_task.sh b/ush/load_modules_run_task.sh index 411f0bdb1..1f0d88126 100755 --- a/ush/load_modules_run_task.sh +++ b/ush/load_modules_run_task.sh @@ -71,30 +71,6 @@ fi # ..." and "module load ..." calls later below that are used to load the # appropriate module file for the specified task. # -# Note that the build of the FV3 forecast model code generates the shell -# script at -# -# ${UFS_WTHR_MDL_DIR}/NEMS/src/conf/module-setup.sh -# -# that can be used to initialize the Lmod (Lua-based module) system/ -# software for handling modules. This script: -# -# 1) Detects the shell in which it is being invoked (i.e. the shell of -# the "parent" script in which it is being sourced). -# 2) Detects the machine it is running on and and calls the appropriate -# (shell- and machine-dependent) initalization script to initialize -# Lmod. -# 3) Purges all modules. -# 4) Uses the "module use ..." command to prepend or append paths to -# Lmod's search path (MODULEPATH). -# -# We could use this module-setup.sh script to initialize Lmod, but since -# it is only found in the forecast model's directory tree, here we pre- -# fer to perform our own initialization. Ideally, there should be one -# module-setup.sh script that is used by all external repos/codes, but -# such a script does not exist. If/when it does, we will consider -# switching to it instead of using the case-statement below. -# #----------------------------------------------------------------------- # print_info_msg "$VERBOSE" " @@ -128,10 +104,14 @@ case "$MACHINE" in ;; # *) - print_err_msg_exit "\ -The script to source to initialize lmod (module loads) has not yet been -specified for the current machine (MACHINE): - MACHINE = \"$MACHINE\"" + if [[ -n ${LMOD_PATH:-""} && -f ${LMOD_PATH:-""} ]] ; then + . ${LMOD_PATH} + else + print_err_msg_exit "\ + The script to source to initialize lmod (module loads) has not yet been + specified for the current machine (MACHINE): + MACHINE = \"$MACHINE\"" + fi ;; # esac @@ -147,13 +127,15 @@ jjob_fp="$2" # #----------------------------------------------------------------------- # -# Sourcing ufs-srweather-app README file (in directory specified by mod- -# ules_dir) for the specified task +# Sourcing ufs-srweather-app build env file # #----------------------------------------------------------------------- # + +module purge + machine=$(echo_lowercase $MACHINE) -env_fn="build_${machine}_${COMPILER}.env" +env_fn=${BUILD_ENV_FN:-"build_${machine}_${COMPILER}.env"} env_fp="${SR_WX_APP_TOP_DIR}/env/${env_fn}" source "${env_fp}" || print_err_msg_exit "\ Sourcing platform- and compiler-specific environment file (env_fp) for the @@ -172,25 +154,23 @@ workflow task specified by task_name failed: # sets environment variables (including prepending/appending to paths) # and loads modules. # -# The regional_workflow repository contains module files for all the +# The regional_workflow repository contains module files for the # workflow tasks in the template rocoto XML file for the FV3-LAM work- -# flow. The full path to a module file for a given task is +# flow that need modules not loaded in the env_fn above. +# +# The full path to a module file for a given task is # -# $HOMErrfs/modulefiles/$machine/${task_name} +# $HOMErrfs/modulefiles/$machine/${task_name}.local # # where HOMErrfs is the base directory of the workflow, machine is the # name of the machine that we're running on (in lowercase), and task_- -# name is the name of the current task (an input to this script). The -# collection of modulefiles is staged by the generate_workflow.sh -# script. Please see that script for information on their creation. +# name is the name of the current task (an input to this script). # #----------------------------------------------------------------------- # modules_dir="$HOMErrfs/modulefiles/tasks/$machine" modulefile_name="${task_name}" default_modules_dir="$HOMErrfs/modulefiles" -default_modulefile_name="${machine}.default" -use_default_modulefile=0 # #----------------------------------------------------------------------- # @@ -198,46 +178,28 @@ use_default_modulefile=0 # #----------------------------------------------------------------------- # - print_info_msg "$VERBOSE" " + +print_info_msg "$VERBOSE" " Loading modules for task \"${task_name}\" ..." - module use "${modules_dir}" || print_err_msg_exit "\ +module use "${modules_dir}" || print_err_msg_exit "\ Call to \"module use\" command failed." - # - # If NOT using the default modulefile... - # -# if [ ${use_default_modulefile} -eq 0 ]; then -# -# module use -a "${modules_dir}" || print_err_msg_exit "\ -#Call to \"module use\" command failed." -# - # - # Load the .local module file if available for the given task - # - modulefile_local="${task_name}.local" - if [ -f ${modules_dir}/${modulefile_local} ]; then - module load "${modulefile_local}" || print_err_msg_exit "\ -Loading .local module file (in directory specified by mod- -ules_dir) for the specified task (task_name) failed: - task_name = \"${task_name}\" - modulefile_local = \"${modulefile_local}\" - modules_dir = \"${modules_dir}\"" - fi - -# else # using default modulefile # -# module load "${default_modulefile_name}" || print_err_msg_exit "\ -#Loading of default module file failed: -# task_name = \"${task_name}\" -# default_modulefile_name = \"${default_modulefile_name}\" -# default_modules_dir = \"${default_modules_dir}\"" +# Load the .local module file if available for the given task # -# fi +modulefile_local="${task_name}.local" +if [ -f ${modules_dir}/${modulefile_local} ]; then + module load "${modulefile_local}" || print_err_msg_exit "\ + Loading .local module file (in directory specified by mod- + ules_dir) for the specified task (task_name) failed: + task_name = \"${task_name}\" + modulefile_local = \"${modulefile_local}\" + modules_dir = \"${modules_dir}\"" +fi - module list +module list -#fi #End if statement for tasks that load no modules # Modules that use conda and need an environment activated will set the # SRW_ENV variable to the name of the environment to be activated. That diff --git a/ush/setup.sh b/ush/setup.sh index a166fafcc..1dcc748ab 100755 --- a/ush/setup.sh +++ b/ush/setup.sh @@ -1,3 +1,4 @@ +#!/bin/bash # #----------------------------------------------------------------------- # @@ -516,7 +517,7 @@ check_var_valid_value "MACHINE" "valid_vals_MACHINE" # several queues. These queues are defined in the default and local # workflow/experiment configuration script. # -# Also, set the machine-dependent flag RELAITVE_OR_NULL that specifies +# Also, set the machine-dependent flag RELATIVE_OR_NULL that specifies # the flag to pass to the link creation command (ln_vrfy) when attempting # to create relative symlinks. On machines that don't support relative # symlinks, it should be set to a null string. @@ -524,12 +525,11 @@ check_var_valid_value "MACHINE" "valid_vals_MACHINE" #----------------------------------------------------------------------- # RELATIVE_LINK_FLAG="" -NCORES_PER_NODE="2" # Need some arbitrary default value to avoid division by zero errors case $MACHINE in "WCOSS_CRAY") WORKFLOW_MANAGER="rocoto" - NCORES_PER_NODE="24" + NCORES_PER_NODE="${NCORES_PER_NODE:-24}" SCHED="lsfcray" QUEUE_DEFAULT=${QUEUE_DEFAULT:-"dev"} QUEUE_HPSS=${QUEUE_HPSS:-"dev_transfer"} @@ -540,7 +540,7 @@ case $MACHINE in "WCOSS_DELL_P3") WORKFLOW_MANAGER="rocoto" - NCORES_PER_NODE=24 + NCORES_PER_NODE="${NCORES_PER_NODE:-24}" SCHED="lsf" QUEUE_DEFAULT=${QUEUE_DEFAULT:-"dev"} QUEUE_HPSS=${QUEUE_HPSS:-"dev_transfer"} @@ -551,7 +551,7 @@ case $MACHINE in "HERA") WORKFLOW_MANAGER="rocoto" - NCORES_PER_NODE=40 + NCORES_PER_NODE="${NCORES_PER_NODE:-40}" SCHED=${SCHED:-"slurm"} PARTITION_DEFAULT=${PARTITION_DEFAULT:-"hera"} QUEUE_DEFAULT=${QUEUE_DEFAULT:-"batch"} @@ -565,7 +565,7 @@ case $MACHINE in "ORION") WORKFLOW_MANAGER="rocoto" - NCORES_PER_NODE=40 + NCORES_PER_NODE="${NCORES_PER_NODE:-40}" SCHED=${SCHED:-"slurm"} PARTITION_DEFAULT=${PARTITION_DEFAULT:-"orion"} QUEUE_DEFAULT=${QUEUE_DEFAULT:-"batch"} @@ -579,7 +579,7 @@ case $MACHINE in "JET") WORKFLOW_MANAGER="rocoto" - NCORES_PER_NODE=24 + NCORES_PER_NODE="${NCORES_PER_NODE:-24}" SCHED=${SCHED:-"slurm"} PARTITION_DEFAULT=${PARTITION_DEFAULT:-"sjet,vjet,kjet,xjet"} QUEUE_DEFAULT=${QUEUE_DEFAULT:-"batch"} @@ -593,7 +593,7 @@ case $MACHINE in "ODIN") WORKFLOW_MANAGER="rocoto" - NCORES_PER_NODE=24 + NCORES_PER_NODE="${NCORES_PER_NODE:-24}" SCHED=${SCHED:-"slurm"} PARTITION_DEFAULT=${PARTITION_DEFAULT:-"workq"} QUEUE_DEFAULT=${QUEUE_DEFAULT:-"workq"} @@ -607,7 +607,7 @@ case $MACHINE in "CHEYENNE") WORKFLOW_MANAGER="rocoto" - NCORES_PER_NODE=36 + NCORES_PER_NODE="${NCORES_PER_NODE:-36}" SCHED=${SCHED:-"pbspro"} QUEUE_DEFAULT=${QUEUE_DEFAULT:-"regular"} QUEUE_HPSS=${QUEUE_HPSS:-"regular"} @@ -618,7 +618,7 @@ case $MACHINE in "STAMPEDE") WORKFLOW_MANAGER="rocoto" - NCORES_PER_NODE=68 + NCORES_PER_NODE="${NCORES_PER_NODE:-68}" SCHED="slurm" PARTITION_DEFAULT=${PARTITION_DEFAULT:-"normal"} QUEUE_DEFAULT=${QUEUE_DEFAULT:-"normal"} @@ -636,11 +636,26 @@ case $MACHINE in ;; "LINUX") - WORKFLOW_MANAGER="none" - SCHED="none" + WORKFLOW_MANAGER=${WORKFLOW_MANAGER:-"none"} + SCHED=${SCHED:-"none"} + ;; + + "*") + NCORES_PER_NODE="2" # Need some arbitrary default value to avoid division by zero errors + + print_err_msg_exit "\ + You are running on an unknown platform! MACHINE=${MACHINE} is not a valid + choice." ;; esac + +if [ -z "$NCORES_PER_NODE" ]; then + print_err_msg_exit "\ + NCORES_PER_NODE is a required setting for your platform! Please + set it in config.sh. + MACHINE = ${MACHINE}" +fi # #----------------------------------------------------------------------- # @@ -662,9 +677,8 @@ check_var_valid_value "SCHED" "valid_vals_SCHED" # #----------------------------------------------------------------------- # -# If we are using a workflow manager, run some checks. First, -# verify that the ACCOUNT variable is not empty. Second, ensure that the -# custom RUN_CMD variables are not set. +# If we are using a workflow manager check that the ACCOUNT variable is +# not empty. # #----------------------------------------------------------------------- # @@ -675,9 +689,6 @@ The variable ACCOUNT cannot be empty if you are using a workflow manager: ACCOUNT = \"$ACCOUNT\" WORKFLOW_MANAGER = \"$WORKFLOW_MANAGER\"" fi - RUN_CMD_UTILS="" - RUN_CMD_FCST="" - RUN_CMD_POST="" fi # #----------------------------------------------------------------------- @@ -2946,6 +2957,15 @@ fi # #----------------------------------------------------------------------- # +# Because RUN_CMD_FCST can include PE_MEMBER01 (and theoretically other +# variables calculated in this script), delete the first occurrence of it +# in the var_defns file, and write it again at the end. +# +#----------------------------------------------------------------------- +$SED -i '/^RUN_CMD_FCST=/d' $GLOBAL_VAR_DEFNS_FP +# +#----------------------------------------------------------------------- +# # Continue appending variable definitions to the variable definitions # file. # @@ -3044,7 +3064,7 @@ FVCOM_FILE="${FVCOM_FILE}" # NCORES_PER_NODE="${NCORES_PER_NODE}" PE_MEMBER01="${PE_MEMBER01}" -RUN_CMD_FCST="${RUN_CMD_FCST}" +RUN_CMD_FCST=$(eval echo ${RUN_CMD_FCST}) # #----------------------------------------------------------------------- # diff --git a/ush/templates/FV3LAM_wflow.xml b/ush/templates/FV3LAM_wflow.xml index f7e7b9471..cc77b5ad4 100644 --- a/ush/templates/FV3LAM_wflow.xml +++ b/ush/templates/FV3LAM_wflow.xml @@ -414,7 +414,7 @@ MODULES_RUN_TASK_FP script. &RSRV_FCST; &LOAD_MODULES_RUN_TASK_FP; "&RUN_FCST_TN;" "&JOBSDIR;/JREGIONAL_RUN_FCST" - {%- if machine in ["JET", "HERA"] %} + {%- if machine in ["JET", "HERA", "LINUX"] %} {{ ncores_run_fcst }} {{ native_run_fcst }} {%- else %} From 4e4d2cdf912f67569bf644b2d48d3945d021c196 Mon Sep 17 00:00:00 2001 From: "Chan-Hoo.Jeon-NOAA" <60152248+chan-hoo@users.noreply.github.com> Date: Thu, 11 Nov 2021 11:40:37 -0500 Subject: [PATCH 12/16] Add ulimit to the task scripts for Orion (#626) * Add ulimit to make_grid script for orion * change stacksize of make_ics * Add ulimit to run_post script * Remove change for Hera --- scripts/exregional_make_grid.sh | 2 ++ scripts/exregional_make_ics.sh | 1 + scripts/exregional_run_post.sh | 2 ++ ...rid_RRFS_NA_3km_ics_FV3GFS_lbcs_FV3GFS_suite_RRFS_v1alpha.sh | 1 + 4 files changed, 6 insertions(+) diff --git a/scripts/exregional_make_grid.sh b/scripts/exregional_make_grid.sh index 9d4a1d9f6..2b876401d 100755 --- a/scripts/exregional_make_grid.sh +++ b/scripts/exregional_make_grid.sh @@ -111,6 +111,8 @@ case "$MACHINE" in ;; "ORION") + ulimit -s unlimited + ulimit -a APRUN="time" ;; diff --git a/scripts/exregional_make_ics.sh b/scripts/exregional_make_ics.sh index dd64a4999..c82a987ad 100755 --- a/scripts/exregional_make_ics.sh +++ b/scripts/exregional_make_ics.sh @@ -105,6 +105,7 @@ case "$MACHINE" in "ORION") ulimit -s unlimited + ulimit -a APRUN="srun" ;; diff --git a/scripts/exregional_run_post.sh b/scripts/exregional_run_post.sh index 9b0f26a42..2632d7d55 100755 --- a/scripts/exregional_run_post.sh +++ b/scripts/exregional_run_post.sh @@ -125,6 +125,8 @@ case "$MACHINE" in ;; "ORION") + ulimit -s unlimited + ulimit -a APRUN="srun" ;; diff --git a/tests/WE2E/test_configs/grids_extrn_mdls_suites_community/config.grid_RRFS_NA_3km_ics_FV3GFS_lbcs_FV3GFS_suite_RRFS_v1alpha.sh b/tests/WE2E/test_configs/grids_extrn_mdls_suites_community/config.grid_RRFS_NA_3km_ics_FV3GFS_lbcs_FV3GFS_suite_RRFS_v1alpha.sh index a01a30807..c088805de 100644 --- a/tests/WE2E/test_configs/grids_extrn_mdls_suites_community/config.grid_RRFS_NA_3km_ics_FV3GFS_lbcs_FV3GFS_suite_RRFS_v1alpha.sh +++ b/tests/WE2E/test_configs/grids_extrn_mdls_suites_community/config.grid_RRFS_NA_3km_ics_FV3GFS_lbcs_FV3GFS_suite_RRFS_v1alpha.sh @@ -48,6 +48,7 @@ WTIME_MAKE_LBCS="01:00:00" NNODES_RUN_POST="6" PPN_RUN_POST="12" +OMP_STACKSIZE_MAKE_ICS="2048m" OMP_STACKSIZE_RUN_FCST="2048m" ############################################################################### From 023639bc0ebfdb94516918bf1ca4119f1b055ba3 Mon Sep 17 00:00:00 2001 From: "Chan-Hoo.Jeon-NOAA" <60152248+chan-hoo@users.noreply.github.com> Date: Thu, 11 Nov 2021 13:22:23 -0500 Subject: [PATCH 13/16] Modify WE2E tests for NCO mode (#629) * Modify nco cases of WE2E * Add nco we2e tests for v16 and hrrr suites --- ...id_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16.sh | 26 ------------------- ..._ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v15p2.sh | 26 +++++++++++++++++++ ..._ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v15p2.sh | 26 +++++++++++++++++++ ...25km_ics_HRRR_lbcs_RAP_suite_GFS_v15p2.sh} | 6 ++--- ...US_25km_ics_HRRR_lbcs_RAP_suite_GSD_SAR.sh | 25 ------------------ ..._ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v15p2.sh | 7 +++-- ...m_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16.sh} | 15 ++++++----- ...NUS_3km_ics_HRRR_lbcs_RAP_suite_GSD_SAR.sh | 25 ------------------ ...NUS_3km_ics_HRRR_lbcs_RAP_suite_GSD_SAR.sh | 25 ------------------ 9 files changed, 66 insertions(+), 115 deletions(-) delete mode 100644 tests/WE2E/test_configs/grids_extrn_mdls_suites_nco/config.nco_grid_CONUS_25km_GFDLgrid_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16.sh create mode 100644 tests/WE2E/test_configs/grids_extrn_mdls_suites_nco/config.nco_grid_RRFS_CONUS_13km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v15p2.sh create mode 100644 tests/WE2E/test_configs/grids_extrn_mdls_suites_nco/config.nco_grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v15p2.sh rename tests/WE2E/test_configs/grids_extrn_mdls_suites_nco/{config.nco_grid_RRFS_CONUS_3km_ics_HRRR_lbcs_RAP_suite_HRRR.sh => config.nco_grid_RRFS_CONUS_25km_ics_HRRR_lbcs_RAP_suite_GFS_v15p2.sh} (75%) delete mode 100644 tests/WE2E/test_configs/grids_extrn_mdls_suites_nco/config.nco_grid_RRFS_CONUS_25km_ics_HRRR_lbcs_RAP_suite_GSD_SAR.sh rename tests/WE2E/test_configs/grids_extrn_mdls_suites_nco/{config.nco_grid_RRFS_CONUS_3km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_2017_gfdlmp_regional.sh => config.nco_grid_RRFS_CONUS_3km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16.sh} (52%) delete mode 100644 tests/WE2E/test_configs/grids_extrn_mdls_suites_nco/config.nco_grid_RRFS_CONUS_3km_ics_HRRR_lbcs_RAP_suite_GSD_SAR.sh delete mode 100644 tests/WE2E/test_configs/grids_extrn_mdls_suites_nco/config.nco_grid_RRFS_SUBCONUS_3km_ics_HRRR_lbcs_RAP_suite_GSD_SAR.sh diff --git a/tests/WE2E/test_configs/grids_extrn_mdls_suites_nco/config.nco_grid_CONUS_25km_GFDLgrid_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16.sh b/tests/WE2E/test_configs/grids_extrn_mdls_suites_nco/config.nco_grid_CONUS_25km_GFDLgrid_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16.sh deleted file mode 100644 index 3431f7cdc..000000000 --- a/tests/WE2E/test_configs/grids_extrn_mdls_suites_nco/config.nco_grid_CONUS_25km_GFDLgrid_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16.sh +++ /dev/null @@ -1,26 +0,0 @@ -# -# TEST PURPOSE/DESCRIPTION: -# ------------------------ -# -# This test is to ensure that the workflow running in nco mode completes -# successfully on the CONUS_25km_GFDLgrid grid (which is a GFDLgrid type -# of grid) using the GFS_v16 physics suite with ICs and LBCs derived from -# the FV3GFS. -# - -RUN_ENVIR="nco" -PREEXISTING_DIR_METHOD="rename" - -PREDEF_GRID_NAME="CONUS_25km_GFDLgrid" -CCPP_PHYS_SUITE="FV3_GFS_v16" - -EXTRN_MDL_NAME_ICS="FV3GFS" -EXTRN_MDL_NAME_LBCS="FV3GFS" -USE_USER_STAGED_EXTRN_FILES="TRUE" - -DATE_FIRST_CYCL="20190901" -DATE_LAST_CYCL="20190901" -CYCL_HRS=( "18" ) - -FCST_LEN_HRS="6" -LBC_SPEC_INTVL_HRS="3" diff --git a/tests/WE2E/test_configs/grids_extrn_mdls_suites_nco/config.nco_grid_RRFS_CONUS_13km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v15p2.sh b/tests/WE2E/test_configs/grids_extrn_mdls_suites_nco/config.nco_grid_RRFS_CONUS_13km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v15p2.sh new file mode 100644 index 000000000..141a9857c --- /dev/null +++ b/tests/WE2E/test_configs/grids_extrn_mdls_suites_nco/config.nco_grid_RRFS_CONUS_13km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v15p2.sh @@ -0,0 +1,26 @@ +# +# TEST PURPOSE/DESCRIPTION: +# ------------------------ +# +# This test is to ensure that the workflow running in nco mode completes +# successfully on the RRFS_CONUS_13km grid using the GFS_v15p2 physics +# suite with ICs and LBCs derived from the FV3GFS. +# + +RUN_ENVIR="nco" +PREEXISTING_DIR_METHOD="rename" + +PREDEF_GRID_NAME="RRFS_CONUS_13km" +CCPP_PHYS_SUITE="FV3_GFS_v15p2" + +EXTRN_MDL_NAME_ICS="FV3GFS" +FV3GFS_FILE_FMT_ICS="grib2" +EXTRN_MDL_NAME_LBCS="FV3GFS" +FV3GFS_FILE_FMT_LBCS="grib2" + +DATE_FIRST_CYCL="20190615" +DATE_LAST_CYCL="20190615" +CYCL_HRS=( "00" ) + +FCST_LEN_HRS="6" +LBC_SPEC_INTVL_HRS="3" diff --git a/tests/WE2E/test_configs/grids_extrn_mdls_suites_nco/config.nco_grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v15p2.sh b/tests/WE2E/test_configs/grids_extrn_mdls_suites_nco/config.nco_grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v15p2.sh new file mode 100644 index 000000000..1b7d92162 --- /dev/null +++ b/tests/WE2E/test_configs/grids_extrn_mdls_suites_nco/config.nco_grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v15p2.sh @@ -0,0 +1,26 @@ +# +# TEST PURPOSE/DESCRIPTION: +# ------------------------ +# +# This test is to ensure that the workflow running in nco mode completes +# successfully on the RRFS_CONUS_25km grid using the GFS_v15p2 physics +# suite with ICs and LBCs derived from the FV3GFS. +# + +RUN_ENVIR="nco" +PREEXISTING_DIR_METHOD="rename" + +PREDEF_GRID_NAME="RRFS_CONUS_25km" +CCPP_PHYS_SUITE="FV3_GFS_v15p2" + +EXTRN_MDL_NAME_ICS="FV3GFS" +FV3GFS_FILE_FMT_ICS="grib2" +EXTRN_MDL_NAME_LBCS="FV3GFS" +FV3GFS_FILE_FMT_LBCS="grib2" + +DATE_FIRST_CYCL="20190615" +DATE_LAST_CYCL="20190615" +CYCL_HRS=( "00" ) + +FCST_LEN_HRS="6" +LBC_SPEC_INTVL_HRS="3" diff --git a/tests/WE2E/test_configs/grids_extrn_mdls_suites_nco/config.nco_grid_RRFS_CONUS_3km_ics_HRRR_lbcs_RAP_suite_HRRR.sh b/tests/WE2E/test_configs/grids_extrn_mdls_suites_nco/config.nco_grid_RRFS_CONUS_25km_ics_HRRR_lbcs_RAP_suite_GFS_v15p2.sh similarity index 75% rename from tests/WE2E/test_configs/grids_extrn_mdls_suites_nco/config.nco_grid_RRFS_CONUS_3km_ics_HRRR_lbcs_RAP_suite_HRRR.sh rename to tests/WE2E/test_configs/grids_extrn_mdls_suites_nco/config.nco_grid_RRFS_CONUS_25km_ics_HRRR_lbcs_RAP_suite_GFS_v15p2.sh index 890395ba8..656a4d0da 100644 --- a/tests/WE2E/test_configs/grids_extrn_mdls_suites_nco/config.nco_grid_RRFS_CONUS_3km_ics_HRRR_lbcs_RAP_suite_HRRR.sh +++ b/tests/WE2E/test_configs/grids_extrn_mdls_suites_nco/config.nco_grid_RRFS_CONUS_25km_ics_HRRR_lbcs_RAP_suite_GFS_v15p2.sh @@ -3,15 +3,15 @@ # ------------------------ # # This test is to ensure that the workflow running in nco mode completes -# successfully on the RRFS_CONUS_3km grid using the HRRR physics suite +# successfully on the RRFS_CONUS_25km grid using the FV3_GFS_v15p2 physics suite # with ICs derived from the HRRR and LBCs derived from the RAP. # RUN_ENVIR="nco" PREEXISTING_DIR_METHOD="rename" -PREDEF_GRID_NAME="RRFS_CONUS_3km" -CCPP_PHYS_SUITE="FV3_HRRR" +PREDEF_GRID_NAME="RRFS_CONUS_25km" +CCPP_PHYS_SUITE="FV3_GFS_v15p2" EXTRN_MDL_NAME_ICS="HRRR" EXTRN_MDL_NAME_LBCS="RAP" diff --git a/tests/WE2E/test_configs/grids_extrn_mdls_suites_nco/config.nco_grid_RRFS_CONUS_25km_ics_HRRR_lbcs_RAP_suite_GSD_SAR.sh b/tests/WE2E/test_configs/grids_extrn_mdls_suites_nco/config.nco_grid_RRFS_CONUS_25km_ics_HRRR_lbcs_RAP_suite_GSD_SAR.sh deleted file mode 100644 index 9bd42fa73..000000000 --- a/tests/WE2E/test_configs/grids_extrn_mdls_suites_nco/config.nco_grid_RRFS_CONUS_25km_ics_HRRR_lbcs_RAP_suite_GSD_SAR.sh +++ /dev/null @@ -1,25 +0,0 @@ -# -# TEST PURPOSE/DESCRIPTION: -# ------------------------ -# -# This test is to ensure that the workflow running in nco mode completes -# successfully on the RRFS_CONUS_25km grid using the GSD_SAR physics -# suite with ICs derived from the HRRR and LBCs derived from the RAP. -# - -RUN_ENVIR="nco" -PREEXISTING_DIR_METHOD="rename" - -PREDEF_GRID_NAME="RRFS_CONUS_25km" -CCPP_PHYS_SUITE="FV3_GSD_SAR" - -EXTRN_MDL_NAME_ICS="HRRR" -EXTRN_MDL_NAME_LBCS="RAP" -USE_USER_STAGED_EXTRN_FILES="TRUE" - -DATE_FIRST_CYCL="20200801" -DATE_LAST_CYCL="20200801" -CYCL_HRS=( "00" ) - -FCST_LEN_HRS="6" -LBC_SPEC_INTVL_HRS="3" diff --git a/tests/WE2E/test_configs/grids_extrn_mdls_suites_nco/config.nco_grid_RRFS_CONUS_3km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v15p2.sh b/tests/WE2E/test_configs/grids_extrn_mdls_suites_nco/config.nco_grid_RRFS_CONUS_3km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v15p2.sh index b0d240892..aac14a479 100644 --- a/tests/WE2E/test_configs/grids_extrn_mdls_suites_nco/config.nco_grid_RRFS_CONUS_3km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v15p2.sh +++ b/tests/WE2E/test_configs/grids_extrn_mdls_suites_nco/config.nco_grid_RRFS_CONUS_3km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v15p2.sh @@ -17,11 +17,10 @@ EXTRN_MDL_NAME_ICS="FV3GFS" FV3GFS_FILE_FMT_ICS="grib2" EXTRN_MDL_NAME_LBCS="FV3GFS" FV3GFS_FILE_FMT_LBCS="grib2" -USE_USER_STAGED_EXTRN_FILES="TRUE" -DATE_FIRST_CYCL="20190901" -DATE_LAST_CYCL="20190901" -CYCL_HRS=( "18" ) +DATE_FIRST_CYCL="20190615" +DATE_LAST_CYCL="20190615" +CYCL_HRS=( "00" ) FCST_LEN_HRS="6" LBC_SPEC_INTVL_HRS="3" diff --git a/tests/WE2E/test_configs/grids_extrn_mdls_suites_nco/config.nco_grid_RRFS_CONUS_3km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_2017_gfdlmp_regional.sh b/tests/WE2E/test_configs/grids_extrn_mdls_suites_nco/config.nco_grid_RRFS_CONUS_3km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16.sh similarity index 52% rename from tests/WE2E/test_configs/grids_extrn_mdls_suites_nco/config.nco_grid_RRFS_CONUS_3km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_2017_gfdlmp_regional.sh rename to tests/WE2E/test_configs/grids_extrn_mdls_suites_nco/config.nco_grid_RRFS_CONUS_3km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16.sh index d8eeef6c5..64c267636 100644 --- a/tests/WE2E/test_configs/grids_extrn_mdls_suites_nco/config.nco_grid_RRFS_CONUS_3km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_2017_gfdlmp_regional.sh +++ b/tests/WE2E/test_configs/grids_extrn_mdls_suites_nco/config.nco_grid_RRFS_CONUS_3km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16.sh @@ -3,23 +3,24 @@ # ------------------------ # # This test is to ensure that the workflow running in nco mode completes -# successfully on the RRFS_CONUS_3km grid using the GFS_2017_gfdlmp_regional -# physics suite with ICs and LBCs derived from the FV3GFS. +# successfully on the RRFS_CONUS_3km grid using the GFS_v16 physics +# suite with ICs and LBCs derived from the FV3GFS. # RUN_ENVIR="nco" PREEXISTING_DIR_METHOD="rename" PREDEF_GRID_NAME="RRFS_CONUS_3km" -CCPP_PHYS_SUITE="FV3_GFS_2017_gfdlmp_regional" +CCPP_PHYS_SUITE="FV3_GFS_v16" EXTRN_MDL_NAME_ICS="FV3GFS" +FV3GFS_FILE_FMT_ICS="grib2" EXTRN_MDL_NAME_LBCS="FV3GFS" -USE_USER_STAGED_EXTRN_FILES="TRUE" +FV3GFS_FILE_FMT_LBCS="grib2" -DATE_FIRST_CYCL="20190901" -DATE_LAST_CYCL="20190901" -CYCL_HRS=( "18" ) +DATE_FIRST_CYCL="20190615" +DATE_LAST_CYCL="20190615" +CYCL_HRS=( "00" ) FCST_LEN_HRS="6" LBC_SPEC_INTVL_HRS="3" diff --git a/tests/WE2E/test_configs/grids_extrn_mdls_suites_nco/config.nco_grid_RRFS_CONUS_3km_ics_HRRR_lbcs_RAP_suite_GSD_SAR.sh b/tests/WE2E/test_configs/grids_extrn_mdls_suites_nco/config.nco_grid_RRFS_CONUS_3km_ics_HRRR_lbcs_RAP_suite_GSD_SAR.sh deleted file mode 100644 index eae6b7401..000000000 --- a/tests/WE2E/test_configs/grids_extrn_mdls_suites_nco/config.nco_grid_RRFS_CONUS_3km_ics_HRRR_lbcs_RAP_suite_GSD_SAR.sh +++ /dev/null @@ -1,25 +0,0 @@ -# -# TEST PURPOSE/DESCRIPTION: -# ------------------------ -# -# This test is to ensure that the workflow running in nco mode completes -# successfully on the RRFS_CONUS_3km grid using the GSD_SAR physics -# suite with ICs derived from the HRRR and LBCs derived from the RAP. -# - -RUN_ENVIR="nco" -PREEXISTING_DIR_METHOD="rename" - -PREDEF_GRID_NAME="RRFS_CONUS_3km" -CCPP_PHYS_SUITE="FV3_GSD_SAR" - -EXTRN_MDL_NAME_ICS="HRRR" -EXTRN_MDL_NAME_LBCS="RAP" -USE_USER_STAGED_EXTRN_FILES="TRUE" - -DATE_FIRST_CYCL="20200801" -DATE_LAST_CYCL="20200801" -CYCL_HRS=( "00" ) - -FCST_LEN_HRS="6" -LBC_SPEC_INTVL_HRS="3" diff --git a/tests/WE2E/test_configs/grids_extrn_mdls_suites_nco/config.nco_grid_RRFS_SUBCONUS_3km_ics_HRRR_lbcs_RAP_suite_GSD_SAR.sh b/tests/WE2E/test_configs/grids_extrn_mdls_suites_nco/config.nco_grid_RRFS_SUBCONUS_3km_ics_HRRR_lbcs_RAP_suite_GSD_SAR.sh deleted file mode 100644 index 3770547ef..000000000 --- a/tests/WE2E/test_configs/grids_extrn_mdls_suites_nco/config.nco_grid_RRFS_SUBCONUS_3km_ics_HRRR_lbcs_RAP_suite_GSD_SAR.sh +++ /dev/null @@ -1,25 +0,0 @@ -# -# TEST PURPOSE/DESCRIPTION: -# ------------------------ -# -# This test is to ensure that the workflow running in nco mode completes -# successfully on the RRFS_SUBCONUS_3km grid using the GSD_SAR physics -# suite with ICs derived from the HRRR and LBCs derived from the RAP. -# - -RUN_ENVIR="nco" -PREEXISTING_DIR_METHOD="rename" - -PREDEF_GRID_NAME="RRFS_SUBCONUS_3km" -CCPP_PHYS_SUITE="FV3_GSD_SAR" - -EXTRN_MDL_NAME_ICS="HRRR" -EXTRN_MDL_NAME_LBCS="RAP" -USE_USER_STAGED_EXTRN_FILES="TRUE" - -DATE_FIRST_CYCL="20200801" -DATE_LAST_CYCL="20200801" -CYCL_HRS=( "00" ) - -FCST_LEN_HRS="6" -LBC_SPEC_INTVL_HRS="3" From bf733c1590fe32b25fb7a5cc942e88b34c8a5e5a Mon Sep 17 00:00:00 2001 From: "Chan-Hoo.Jeon-NOAA" <60152248+chan-hoo@users.noreply.github.com> Date: Mon, 15 Nov 2021 17:17:57 -0500 Subject: [PATCH 14/16] Remove rocoto for wcoss from launch script (#636) --- ush/launch_FV3LAM_wflow.sh | 28 ++++++++-------------------- 1 file changed, 8 insertions(+), 20 deletions(-) diff --git a/ush/launch_FV3LAM_wflow.sh b/ush/launch_FV3LAM_wflow.sh index fe7ec8df6..76c964083 100755 --- a/ush/launch_FV3LAM_wflow.sh +++ b/ush/launch_FV3LAM_wflow.sh @@ -106,26 +106,14 @@ expt_name="${EXPT_SUBDIR}" # #----------------------------------------------------------------------- # -if [ "$MACHINE" = "WCOSS_DELL_P3" ]; then - module purge - module load lsf/10.1 - module use /gpfs/dell3/usrx/local/dev/emc_rocoto/modulefiles/ - module load ruby/2.5.1 rocoto/1.3.0rc2 -elif [ "$MACHINE" = "WCOSS_CRAY" ]; then - module purge - module load xt-lsfhpc/9.1.3 - module use -a /usrx/local/emc_rocoto/modulefiles - module load rocoto/1.3.0rc2 -else - machine=$(echo_lowercase $MACHINE) - env_fn=${WFLOW_ENV_FN:-"wflow_${machine}.env"} - env_fp="${SR_WX_APP_TOP_DIR}/env/${env_fn}" - module purge - source "${env_fp}" || print_err_msg_exit "\ - Sourcing platform-specific environment file (env_fp) for - the workflow task failed : - env_fp = \"${env_fp}\"" -fi +machine=$(echo_lowercase $MACHINE) +env_fn=${WFLOW_ENV_FN:-"wflow_${machine}.env"} +env_fp="${SR_WX_APP_TOP_DIR}/env/${env_fn}" +module purge +source "${env_fp}" || print_err_msg_exit "\ + Sourcing platform-specific environment file (env_fp) for +the workflow task failed : +env_fp = \"${env_fp}\"" # #----------------------------------------------------------------------- # From cbafca3fbca4a510e652979496e92974e3415a49 Mon Sep 17 00:00:00 2001 From: "Chan-Hoo.Jeon-NOAA" <60152248+chan-hoo@users.noreply.github.com> Date: Wed, 17 Nov 2021 12:51:55 -0500 Subject: [PATCH 15/16] Rename unchanged readlink (#638) --- scripts/exregional_run_ensgridvx.sh | 2 +- scripts/exregional_run_ensgridvx_mean.sh | 2 +- scripts/exregional_run_ensgridvx_prob.sh | 2 +- scripts/exregional_run_enspointvx.sh | 2 +- scripts/exregional_run_enspointvx_mean.sh | 2 +- scripts/exregional_run_enspointvx_prob.sh | 2 +- 6 files changed, 6 insertions(+), 6 deletions(-) diff --git a/scripts/exregional_run_ensgridvx.sh b/scripts/exregional_run_ensgridvx.sh index d00d0f0fe..78cb5c2ba 100755 --- a/scripts/exregional_run_ensgridvx.sh +++ b/scripts/exregional_run_ensgridvx.sh @@ -28,7 +28,7 @@ set -x # #----------------------------------------------------------------------- # -scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) scrfunc_fn=$( basename "${scrfunc_fp}" ) scrfunc_dir=$( dirname "${scrfunc_fp}" ) # diff --git a/scripts/exregional_run_ensgridvx_mean.sh b/scripts/exregional_run_ensgridvx_mean.sh index 7629cd669..b255adb86 100755 --- a/scripts/exregional_run_ensgridvx_mean.sh +++ b/scripts/exregional_run_ensgridvx_mean.sh @@ -28,7 +28,7 @@ set -x # #----------------------------------------------------------------------- # -scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) scrfunc_fn=$( basename "${scrfunc_fp}" ) scrfunc_dir=$( dirname "${scrfunc_fp}" ) # diff --git a/scripts/exregional_run_ensgridvx_prob.sh b/scripts/exregional_run_ensgridvx_prob.sh index fc4749d15..2ed9bdfea 100755 --- a/scripts/exregional_run_ensgridvx_prob.sh +++ b/scripts/exregional_run_ensgridvx_prob.sh @@ -28,7 +28,7 @@ set -x # #----------------------------------------------------------------------- # -scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) scrfunc_fn=$( basename "${scrfunc_fp}" ) scrfunc_dir=$( dirname "${scrfunc_fp}" ) # diff --git a/scripts/exregional_run_enspointvx.sh b/scripts/exregional_run_enspointvx.sh index e3985314d..571711bf5 100755 --- a/scripts/exregional_run_enspointvx.sh +++ b/scripts/exregional_run_enspointvx.sh @@ -28,7 +28,7 @@ set -x # #----------------------------------------------------------------------- # -scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) scrfunc_fn=$( basename "${scrfunc_fp}" ) scrfunc_dir=$( dirname "${scrfunc_fp}" ) # diff --git a/scripts/exregional_run_enspointvx_mean.sh b/scripts/exregional_run_enspointvx_mean.sh index cd0d3f636..4cd6039b9 100755 --- a/scripts/exregional_run_enspointvx_mean.sh +++ b/scripts/exregional_run_enspointvx_mean.sh @@ -28,7 +28,7 @@ set -x # #----------------------------------------------------------------------- # -scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) scrfunc_fn=$( basename "${scrfunc_fp}" ) scrfunc_dir=$( dirname "${scrfunc_fp}" ) # diff --git a/scripts/exregional_run_enspointvx_prob.sh b/scripts/exregional_run_enspointvx_prob.sh index fc21f54b8..50d4e606f 100755 --- a/scripts/exregional_run_enspointvx_prob.sh +++ b/scripts/exregional_run_enspointvx_prob.sh @@ -28,7 +28,7 @@ set -x # #----------------------------------------------------------------------- # -scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" ) +scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" ) scrfunc_fn=$( basename "${scrfunc_fp}" ) scrfunc_dir=$( dirname "${scrfunc_fp}" ) # From 68694fcfec4df1946e2d55ff0af65845dbdeab86 Mon Sep 17 00:00:00 2001 From: "Chan-Hoo.Jeon-NOAA" <60152248+chan-hoo@users.noreply.github.com> Date: Wed, 17 Nov 2021 15:32:21 -0500 Subject: [PATCH 16/16] Fix -np warning by RUN_CMD_FCST (#633) * move to easy fix --- ush/setup.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ush/setup.sh b/ush/setup.sh index 1dcc748ab..f00e3bf8d 100755 --- a/ush/setup.sh +++ b/ush/setup.sh @@ -3064,7 +3064,7 @@ FVCOM_FILE="${FVCOM_FILE}" # NCORES_PER_NODE="${NCORES_PER_NODE}" PE_MEMBER01="${PE_MEMBER01}" -RUN_CMD_FCST=$(eval echo ${RUN_CMD_FCST}) +RUN_CMD_FCST="$(eval echo \'${RUN_CMD_FCST}\')" # #----------------------------------------------------------------------- #