From b78d31c27e213bb863d219ec4a5bd9a778a951af Mon Sep 17 00:00:00 2001 From: "Kate.Friedman" Date: Thu, 4 Jan 2024 18:57:33 +0000 Subject: [PATCH 01/18] Update ROTDIR references in documentation - Remove references to COMROT and replace with ROTDIR Refs #2061 --- docs/doxygen/mainpage.h | 12 ++++++------ docs/source/errors_faq.rst | 4 ++-- docs/source/init.rst | 10 +++++----- docs/source/setup.rst | 24 ++++++++++++------------ docs/source/start.rst | 2 +- docs/source/view.rst | 6 +++--- 6 files changed, 29 insertions(+), 29 deletions(-) diff --git a/docs/doxygen/mainpage.h b/docs/doxygen/mainpage.h index 40e8e6f946b..6d455b4904f 100644 --- a/docs/doxygen/mainpage.h +++ b/docs/doxygen/mainpage.h @@ -12,7 +12,7 @@ With the FV3GFS system, a Rocoto driven workflow is being used. This workflow builds on the experiences and work of Kate.Howard and Terry.McGuinness for the GSM, but strips down a lot of complications arising from the use of a cumbersome \c para_config. The \c para_config no longer exists in the workflow and instead a number of \c config files are added, one each for an individual task e.g. \p config.anal contains the analysis specific information. A base config called as \c config.base contains information related to the machine, super-structure, etc. The idea behind splitting the \p para_config into individual, smaller and managable configs is to provide a means to run any chosen task without the overhead of the full cycling framework. All the configs are located under \c fv3gfs/config -Additionally, the structure of the \c COMROT directory is now modified to look like operations. This enables the use of the workflow much closer to the operational environment, with the exception of the workflow manager. +Additionally, the structure of the \c ROTDIR directory is now modified to look like operations. This enables the use of the workflow much closer to the operational environment, with the exception of the workflow manager. This is a very much a work in progress and any issues should be reported back and are greatly appreciated. @@ -23,12 +23,12 @@ To setup an experiment, a python script \c setup_expt.py (located in \ $> setup_expt.py -h usage: setup_expt.py [-h] --pslot PSLOT [--configdir CONFIGDIR] [--idate IDATE] [--icsdir ICSDIR] - [--resdet RESDET] [--resens RESENS] [--comrot COMROT] + [--resdet RESDET] [--resens RESENS] [--comroot COMROOT] [--expdir EXPDIR] [--nens NENS] [--cdump CDUMP] Setup files and directories to start a GFS parallel. Create EXPDIR, copy - config files Create COMROT experiment directory structure, link initial - condition files from $ICSDIR to $COMROT + config files Create ROTDIR experiment directory structure, link initial + condition files from $ICSDIR to $ROTDIR optional arguments: -h, --help show this help message and exit @@ -44,7 +44,7 @@ To setup an experiment, a python script \c setup_expt.py (located in \ (default: 384) --resens resolution of the ensemble model forecast (default: 192) - --comrot full path to COMROT + --comroot full path to COMROOT, where ROTDIR (COMROOT+PSLOT) will be created (default: None) --expdir full path to EXPDIR (default: None) @@ -53,7 +53,7 @@ To setup an experiment, a python script \c setup_expt.py (located in \ --cdump CDUMP to start the experiment (default: gdas) -The above script creates directories \c EXPDIR and \c COMROT. It will make links for initial conditions from a location provided via the \c --icsdir argument for a chosen resolution for the control \c --resdet and the ensemble \c --resens. Experiment name is controlled by the input argument \c --pslot. The script will ask user input in case any of the directories already exist. It will copy experiment configuration files into the \c EXPDIR from \c CONFIGDIR. +The above script creates directories \c EXPDIR and \c ROTDIR. It will make links for initial conditions from a location provided via the \c --icsdir argument for a chosen resolution for the control \c --resdet and the ensemble \c --resens. Experiment name is controlled by the input argument \c --pslot. The script will ask user input in case any of the directories already exist. It will copy experiment configuration files into the \c EXPDIR from \c CONFIGDIR. Sample initial conditions for a few resolutions are available at:
Hera: TODO: /path/here/for/initial/conditions
diff --git a/docs/source/errors_faq.rst b/docs/source/errors_faq.rst index 2660a01e609..519e29bacec 100644 --- a/docs/source/errors_faq.rst +++ b/docs/source/errors_faq.rst @@ -36,10 +36,10 @@ Example:: **Solution:** set TERM to "xterm" (bash: export TERM=xterm ; csh/tcsh: setenv TERM xterm) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Issue: Directory name change for EnKF folder in COMROT +Issue: Directory name change for EnKF folder in ROTDIR ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -**Issue:** The EnKF COMROT folders were renamed during the GFS v15 development process to remove the period between "enkf" and "gdas": enkf.gdas.$PDY → enkfgdas.$PDY +**Issue:** The EnKF ROTDIR folders were renamed during the GFS v15 development process to remove the period between "enkf" and "gdas": enkf.gdas.$PDY → enkfgdas.$PDY **Fix:** Older tarballs on HPSS will have the older directory name with the period between 'enkf' and 'gdas'. Make sure to rename folder to 'enkfgdas.$PDY' after obtaining. Only an issue for the initial cycle. diff --git a/docs/source/init.rst b/docs/source/init.rst index 65e400c68ef..14a0ea0d564 100644 --- a/docs/source/init.rst +++ b/docs/source/init.rst @@ -391,7 +391,7 @@ The chgres jobs will have a dependency on the data-pull jobs and will wait to ru 5. Check output: -In the config you will have defined an output folder called ``$OUTDIR``. The converted output will be found there, including the needed abias and radstat initial condition files (if CDUMP=gdas). The files will be in the needed directory structure for the global-workflow system, therefore a user can move the contents of their ``$OUTDIR`` directly into their ``$ROTDIR/$COMROT``. +In the config you will have defined an output folder called ``$OUTDIR``. The converted output will be found there, including the needed abias and radstat initial condition files (if CDUMP=gdas). The files will be in the needed directory structure for the global-workflow system, therefore a user can move the contents of their ``$OUTDIR`` directly into their ``$ROTDIR``. Please report bugs to George Gayno (george.gayno@noaa.gov) and Kate Friedman (kate.friedman@noaa.gov). @@ -449,7 +449,7 @@ The warm starts and other output from production are at C768 deterministic and C What files should you pull for starting a new experiment with warm starts from production? ------------------------------------------------------------------------------------------ -That depends on what mode you want to run -- forecast-only or cycled. Whichever mode, navigate to the top of your ``COMROT`` and pull the entirety of the tarball(s) listed below for your mode. The files within the tarball are already in the ``$CDUMP.$PDY/$CYC/$ATMOS`` folder format expected by the system. +That depends on what mode you want to run -- forecast-only or cycled. Whichever mode, navigate to the top of your ``ROTDIR`` and pull the entirety of the tarball(s) listed below for your mode. The files within the tarball are already in the ``$CDUMP.$PDY/$CYC/$ATMOS`` folder format expected by the system. For forecast-only there are two tarballs to pull @@ -489,7 +489,7 @@ Tarballs per cycle: com_gfs_vGFSVER_enkfgdas.YYYYMMDD_CC.enkfgdas_restart_grp7.tar com_gfs_vGFSVER_enkfgdas.YYYYMMDD_CC.enkfgdas_restart_grp8.tar -Go to the top of your ``COMROT/ROTDIR`` and pull the contents of all tarballs there. The tarballs already contain the needed directory structure. +Go to the top of your ``ROTDIR`` and pull the contents of all tarballs there. The tarballs already contain the needed directory structure. .. _warmstarts-preprod-parallels: @@ -514,7 +514,7 @@ Recent pre-implementation parallel series was for GFS v16 (implemented March 202 + ../$GDATE/gdas_restartb.tar + ../$GDATE/enkfgdas_restartb_grp##.tar (where ## is 01 through 08) (note, older tarballs may include a period between enkf and gdas: "enkf.gdas") -* **Where do I put the warm-start initial conditions?** Extraction should occur right inside your COMROT. You may need to rename the enkf folder (enkf.gdas.$PDY -> enkfgdas.$PDY). +* **Where do I put the warm-start initial conditions?** Extraction should occur right inside your ROTDIR. You may need to rename the enkf folder (enkf.gdas.$PDY -> enkfgdas.$PDY). Due to a recent change in the dycore, you may also need an additional offline step to fix the checksum of the NetCDF files for warm start. See the :ref:`Fix netcdf checksum section `. @@ -602,5 +602,5 @@ And then on all platforms: :: - cd $COMROT + cd $ROTDIR for f in $(find ./ -name *tile*.nc); do echo $f; ncatted -a checksum,,d,, $f; done diff --git a/docs/source/setup.rst b/docs/source/setup.rst index be04aa5d96c..c86f8ce95aa 100644 --- a/docs/source/setup.rst +++ b/docs/source/setup.rst @@ -29,7 +29,7 @@ The following command examples include variables for reference but users should cd workflow ./setup_expt.py gfs forecast-only --idate $IDATE --edate $EDATE [--app $APP] [--start $START] [--gfs_cyc $GFS_CYC] [--resdet $RESDET] - [--pslot $PSLOT] [--configdir $CONFIGDIR] [--comrot $COMROT] [--expdir $EXPDIR] + [--pslot $PSLOT] [--configdir $CONFIGDIR] [--comroot $COMROOT] [--expdir $EXPDIR] where: @@ -52,7 +52,7 @@ where: * ``$CONFIGDIR`` is the path to the ``/config`` folder under the copy of the system you're using [default: $TOP_OF_CLONE/parm/config/] * ``$RESDET`` is the FV3 resolution (i.e. 768 for C768) [default: 384] * ``$GFS_CYC`` is the forecast frequency (0 = none, 1 = 00z only [default], 2 = 00z & 12z, 4 = all cycles) - * ``$COMROT`` is the path to your experiment output directory. DO NOT include PSLOT folder at end of path, it’ll be built for you. [default: $HOME (but do not use default due to limited space in home directories normally, provide a path to a larger scratch space)] + * ``$COMROOT`` is the path to your experiment output directory. Your ``ROTDIR`` (rotating com directory) will be created using ``COMROOT`` and ``PSLOT``. [default: $HOME (but do not use default due to limited space in home directories normally, provide a path to a larger scratch space)] * ``$EXPDIR`` is the path to your experiment directory where your configs will be placed and where you will find your workflow monitoring files (i.e. rocoto database and xml file). DO NOT include PSLOT folder at end of path, it will be built for you. [default: $HOME] Examples: @@ -62,21 +62,21 @@ Atm-only: :: cd workflow - ./setup_expt.py gfs forecast-only --pslot test --idate 2020010100 --edate 2020010118 --resdet 384 --gfs_cyc 4 --comrot /some_large_disk_area/Joe.Schmo/comrot --expdir /some_safe_disk_area/Joe.Schmo/expdir + ./setup_expt.py gfs forecast-only --pslot test --idate 2020010100 --edate 2020010118 --resdet 384 --gfs_cyc 4 --comroot /some_large_disk_area/Joe.Schmo/comroot --expdir /some_safe_disk_area/Joe.Schmo/expdir Coupled: :: cd workflow - ./setup_expt.py gfs forecast-only --app S2SW --pslot coupled_test --idate 2013040100 --edate 2013040100 --resdet 384 --comrot /some_large_disk_area/Joe.Schmo/comrot --expdir /some_safe_disk_area/Joe.Schmo/expdir + ./setup_expt.py gfs forecast-only --app S2SW --pslot coupled_test --idate 2013040100 --edate 2013040100 --resdet 384 --comroot /some_large_disk_area/Joe.Schmo/comroot --expdir /some_safe_disk_area/Joe.Schmo/expdir Coupled with aerosols: :: cd workflow - ./setup_expt.py gfs forecast-only --app S2SWA --pslot coupled_test --idate 2013040100 --edate 2013040100 --resdet 384 --comrot /some_large_disk_area/Joe.Schmo/comrot --expdir /some_safe_disk_area/Joe.Schmo/expdir + ./setup_expt.py gfs forecast-only --app S2SWA --pslot coupled_test --idate 2013040100 --edate 2013040100 --resdet 384 --comroot /some_large_disk_area/Joe.Schmo/comroot --expdir /some_safe_disk_area/Joe.Schmo/expdir **************************************** Step 2: Set user and experiment settings @@ -141,7 +141,7 @@ The following command examples include variables for reference but users should cd workflow ./setup_expt.py gfs cycled --idate $IDATE --edate $EDATE [--app $APP] [--start $START] [--gfs_cyc $GFS_CYC] [--resdet $RESDET] [--resens $RESENS] [--nens $NENS] [--cdump $CDUMP] - [--pslot $PSLOT] [--configdir $CONFIGDIR] [--comrot $COMROT] [--expdir $EXPDIR] [--icsdir $ICSDIR] + [--pslot $PSLOT] [--configdir $CONFIGDIR] [--comroot $COMROOT] [--expdir $EXPDIR] [--icsdir $ICSDIR] where: @@ -167,7 +167,7 @@ where: * ``$CDUMP`` is the starting phase [default: gdas] * ``$PSLOT`` is the name of your experiment [default: test] * ``$CONFIGDIR`` is the path to the config folder under the copy of the system you're using [default: $TOP_OF_CLONE/parm/config/] - * ``$COMROT`` is the path to your experiment output directory. DO NOT include PSLOT folder at end of path, it’ll be built for you. [default: $HOME] + * ``$COMROOT`` is the path to your experiment output directory. Your ``ROTDIR`` (rotating com directory) will be created using ``COMROOT`` and ``PSLOT``. [default: $HOME] * ``$EXPDIR`` is the path to your experiment directory where your configs will be placed and where you will find your workflow monitoring files (i.e. rocoto database and xml file). DO NOT include PSLOT folder at end of path, it will be built for you. [default: $HOME] * ``$ICSDIR`` is the path to the ICs for your run if generated separately. [default: None] @@ -178,13 +178,13 @@ Example: :: cd workflow - ./setup_expt.py gfs cycled --pslot test --configdir /home/Joe.Schmo/git/global-workflow/parm/config --idate 2020010100 --edate 2020010118 --comrot /some_large_disk_area/Joe.Schmo/comrot --expdir /some_safe_disk_area/Joe.Schmo/expdir --resdet 384 --resens 192 --nens 80 --gfs_cyc 4 + ./setup_expt.py gfs cycled --pslot test --configdir /home/Joe.Schmo/git/global-workflow/parm/config --idate 2020010100 --edate 2020010118 --comroot /some_large_disk_area/Joe.Schmo/comroot --expdir /some_safe_disk_area/Joe.Schmo/expdir --resdet 384 --resens 192 --nens 80 --gfs_cyc 4 Example ``setup_expt.py`` on Orion: :: - Orion-login-3$ ./setup_expt.py gfs cycled --pslot test --idate 2022010118 --edate 2022010200 --resdet 192 --resens 96 --nens 80 --comrot /work/noaa/stmp/jschmo/comrot --expdir /work/noaa/global/jschmo/expdir + Orion-login-3$ ./setup_expt.py gfs cycled --pslot test --idate 2022010118 --edate 2022010200 --resdet 192 --resens 96 --nens 80 --comroot /work/noaa/stmp/jschmo/comroot --expdir /work/noaa/global/jschmo/expdir EDITED: /work/noaa/global/jschmo/expdir/test/config.base as per user input. EDITED: /work/noaa/global/jschmo/expdir/test/config.aeroanl as per user input. EDITED: /work/noaa/global/jschmo/expdir/test/config.ocnanal as per user input. @@ -195,9 +195,9 @@ What happens if I run ``setup_expt.py`` again for an experiment that already exi :: - Orion-login-3$ ./setup_expt.py gfs cycled --pslot test --idate 2022010118 --edate 2022010200 --resdet 192 --resens 96 --nens 80 --comrot /work/noaa/stmp/jschmo/comrot --expdir /work/noaa/global/jschmo/expdir + Orion-login-3$ ./setup_expt.py gfs cycled --pslot test --idate 2022010118 --edate 2022010200 --resdet 192 --resens 96 --nens 80 --comroot /work/noaa/stmp/jschmo/comroot --expdir /work/noaa/global/jschmo/expdir - directory already exists in /work/noaa/stmp/jschmo/comrot/test + directory already exists in /work/noaa/stmp/jschmo/comroot/test Do you wish to over-write [y/N]: y @@ -208,7 +208,7 @@ What happens if I run ``setup_expt.py`` again for an experiment that already exi EDITED: /work/noaa/global/jschmo/expdir/test/config.aeroanl as per user input. EDITED: /work/noaa/global/jschmo/expdir/test/config.ocnanal as per user input. -Your ``COMROT`` and ``EXPDIR`` will be deleted and remade. Be careful with this! +Your ``ROTDIR`` and ``EXPDIR`` will be deleted and remade. Be careful with this! **************************************** Step 2: Set user and experiment settings diff --git a/docs/source/start.rst b/docs/source/start.rst index 7dc093e8a47..fb3c3ef316c 100644 --- a/docs/source/start.rst +++ b/docs/source/start.rst @@ -16,7 +16,7 @@ Start your run from within your EXPDIR The first jobs of your run should now be queued or already running (depending on machine traffic). How exciting! -You'll now have a "logs" folder in both your COMROT and EXPDIR. The EXPDIR log folder contains workflow log files (e.g. rocoto command results) and the COMROT log folder will contain logs for each job (previously known as dayfiles). +You'll now have a "logs" folder in both your ``ROTDIR`` and ``EXPDIR``. The EXPDIR log folder contains workflow log files (e.g. rocoto command results) and the ``ROTDIR`` log folder will contain logs for each job (previously known as dayfiles). ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Set up your experiment cron diff --git a/docs/source/view.rst b/docs/source/view.rst index 3093755e9a4..e321936274a 100644 --- a/docs/source/view.rst +++ b/docs/source/view.rst @@ -2,7 +2,7 @@ View Experiment output ====================== -The output from your run will be found in the ``COMROT/ROTDIR`` you established. This is also where you placed your initial conditions. Within your ``COMROT`` you will have the following directory structure (based on the type of experiment you run): +The output from your run will be found in the ``ROTDIR`` you established. This is also where you placed your initial conditions. Within your ``ROTDIR`` you will have the following directory structure (based on the type of experiment you run): ^^^^^^^^^^^^^ Forecast-only @@ -29,11 +29,11 @@ Cycled logs/ <- logs for each cycle in the run vrfyarch/ <- contains files related to verification and archival -Here is an example ``COMROT`` for a cycled run as it may look several cycles in (note the archival steps remove older cycle folders as the run progresses): +Here is an example ``ROTDIR`` for a cycled run as it may look several cycles in (note the archival steps remove older cycle folders as the run progresses): :: - -bash-4.2$ ll /scratch1/NCEPDEV/stmp4/Joe.Schmo/comrot/testcyc192 + -bash-4.2$ ll /scratch1/NCEPDEV/stmp4/Joe.Schmo/comroot/testcyc192 total 88 drwxr-sr-x 4 Joe.Schmo stmp 4096 Oct 22 04:50 enkfgdas.20190529 drwxr-sr-x 4 Joe.Schmo stmp 4096 Oct 22 07:20 enkfgdas.20190530 From 9163c230634294cab61d260a004fe9daa39c6381 Mon Sep 17 00:00:00 2001 From: "Kate.Friedman" Date: Thu, 4 Jan 2024 18:59:00 +0000 Subject: [PATCH 02/18] Update hosts files to remove COMROOT - Remove COMROOT setting for each platform - Replace COMROOT with full path in COMINsyn Refs #2061 --- workflow/hosts/awspw.yaml | 3 +-- workflow/hosts/container.yaml | 1 - workflow/hosts/hera.yaml | 3 +-- workflow/hosts/hercules.yaml | 3 +-- workflow/hosts/jet.yaml | 3 +-- workflow/hosts/orion.yaml | 3 +-- workflow/hosts/s4.yaml | 3 +-- workflow/hosts/wcoss2.yaml | 3 +-- 8 files changed, 7 insertions(+), 15 deletions(-) diff --git a/workflow/hosts/awspw.yaml b/workflow/hosts/awspw.yaml index becb38e2366..c683010e0e4 100644 --- a/workflow/hosts/awspw.yaml +++ b/workflow/hosts/awspw.yaml @@ -1,8 +1,7 @@ BASE_GIT: '/scratch1/NCEPDEV/global/glopara/git' #TODO: This does not yet exist. DMPDIR: '/scratch1/NCEPDEV/global/glopara/dump' # TODO: This does not yet exist. PACKAGEROOT: '/scratch1/NCEPDEV/global/glopara/nwpara' #TODO: This does not yet exist. -COMROOT: '/scratch1/NCEPDEV/global/glopara/com' #TODO: This does not yet exist. -COMINsyn: '${COMROOT}/gfs/prod/syndat' #TODO: This does not yet exist. +COMINsyn: '/scratch1/NCEPDEV/global/glopara/com/gfs/prod/syndat' #TODO: This does not yet exist. HOMEDIR: '/contrib/${USER}' STMP: '/lustre/${USER}/stmp2/' PTMP: '/lustre/${USER}/stmp4/' diff --git a/workflow/hosts/container.yaml b/workflow/hosts/container.yaml index 879be0bf316..3fd3856679c 100644 --- a/workflow/hosts/container.yaml +++ b/workflow/hosts/container.yaml @@ -1,7 +1,6 @@ BASE_GIT: '' DMPDIR: '/home/${USER}' PACKAGEROOT: '' -COMROOT: '' COMINsyn: '' HOMEDIR: '/home/${USER}' STMP: '/home/${USER}' diff --git a/workflow/hosts/hera.yaml b/workflow/hosts/hera.yaml index e44c0aece08..71d444ed005 100644 --- a/workflow/hosts/hera.yaml +++ b/workflow/hosts/hera.yaml @@ -2,8 +2,7 @@ BASE_GIT: '/scratch1/NCEPDEV/global/glopara/git' DMPDIR: '/scratch1/NCEPDEV/global/glopara/dump' BASE_CPLIC: '/scratch1/NCEPDEV/global/glopara/data/ICSDIR/prototype_ICs' PACKAGEROOT: '/scratch1/NCEPDEV/global/glopara/nwpara' -COMROOT: '/scratch1/NCEPDEV/global/glopara/com' -COMINsyn: '${COMROOT}/gfs/prod/syndat' +COMINsyn: '/scratch1/NCEPDEV/global/glopara/com/gfs/prod/syndat' HOMEDIR: '/scratch1/NCEPDEV/global/${USER}' STMP: '/scratch1/NCEPDEV/stmp2/${USER}' PTMP: '/scratch1/NCEPDEV/stmp4/${USER}' diff --git a/workflow/hosts/hercules.yaml b/workflow/hosts/hercules.yaml index e977091ba6c..58a9589f2f6 100644 --- a/workflow/hosts/hercules.yaml +++ b/workflow/hosts/hercules.yaml @@ -2,8 +2,7 @@ BASE_GIT: '/work/noaa/global/glopara/git' DMPDIR: '/work/noaa/rstprod/dump' BASE_CPLIC: '/work/noaa/global/glopara/data/ICSDIR/prototype_ICs' PACKAGEROOT: '/work/noaa/global/glopara/nwpara' -COMROOT: '/work/noaa/global/glopara/com' -COMINsyn: '${COMROOT}/gfs/prod/syndat' +COMINsyn: '/work/noaa/global/glopara/com/gfs/prod/syndat' HOMEDIR: '/work/noaa/global/${USER}' STMP: '/work/noaa/stmp/${USER}' PTMP: '/work/noaa/stmp/${USER}' diff --git a/workflow/hosts/jet.yaml b/workflow/hosts/jet.yaml index 313ce38dc29..00c89b60a16 100644 --- a/workflow/hosts/jet.yaml +++ b/workflow/hosts/jet.yaml @@ -2,8 +2,7 @@ BASE_GIT: '/lfs4/HFIP/hfv3gfs/glopara/git' DMPDIR: '/lfs4/HFIP/hfv3gfs/glopara/dump' BASE_CPLIC: '/mnt/lfs4/HFIP/hfv3gfs/glopara/data/ICSDIR/prototype_ICs' PACKAGEROOT: '/lfs4/HFIP/hfv3gfs/glopara/nwpara' -COMROOT: '/lfs4/HFIP/hfv3gfs/glopara/com' -COMINsyn: '${COMROOT}/gfs/prod/syndat' +COMINsyn: '/lfs4/HFIP/hfv3gfs/glopara/com/gfs/prod/syndat' HOMEDIR: '/lfs4/HFIP/hfv3gfs/${USER}' STMP: '/lfs4/HFIP/hfv3gfs/${USER}/stmp' PTMP: '/lfs4/HFIP/hfv3gfs/${USER}/ptmp' diff --git a/workflow/hosts/orion.yaml b/workflow/hosts/orion.yaml index 459aee7cf65..4c08a878dcf 100644 --- a/workflow/hosts/orion.yaml +++ b/workflow/hosts/orion.yaml @@ -2,8 +2,7 @@ BASE_GIT: '/work/noaa/global/glopara/git' DMPDIR: '/work/noaa/rstprod/dump' BASE_CPLIC: '/work/noaa/global/glopara/data/ICSDIR/prototype_ICs' PACKAGEROOT: '/work/noaa/global/glopara/nwpara' -COMROOT: '/work/noaa/global/glopara/com' -COMINsyn: '${COMROOT}/gfs/prod/syndat' +COMINsyn: '/work/noaa/global/glopara/com/gfs/prod/syndat' HOMEDIR: '/work/noaa/global/${USER}' STMP: '/work/noaa/stmp/${USER}' PTMP: '/work/noaa/stmp/${USER}' diff --git a/workflow/hosts/s4.yaml b/workflow/hosts/s4.yaml index 01b28c1e6bd..52a9f7a3650 100644 --- a/workflow/hosts/s4.yaml +++ b/workflow/hosts/s4.yaml @@ -2,8 +2,7 @@ BASE_GIT: '/data/prod/glopara/git' DMPDIR: '/data/prod/glopara/dump' BASE_CPLIC: '/data/prod/glopara/coupled_ICs' PACKAGEROOT: '/data/prod/glopara/nwpara' -COMROOT: '/data/prod/glopara/com' -COMINsyn: '${COMROOT}/gfs/prod/syndat' +COMINsyn: '/data/prod/glopara/com/gfs/prod/syndat' HOMEDIR: '/data/users/${USER}' STMP: '/scratch/users/${USER}' PTMP: '/scratch/users/${USER}' diff --git a/workflow/hosts/wcoss2.yaml b/workflow/hosts/wcoss2.yaml index 04a5949b2e7..cfb141061c0 100644 --- a/workflow/hosts/wcoss2.yaml +++ b/workflow/hosts/wcoss2.yaml @@ -2,8 +2,7 @@ BASE_GIT: '/lfs/h2/emc/global/save/emc.global/git' DMPDIR: '/lfs/h2/emc/dump/noscrub/dump' BASE_CPLIC: '/lfs/h2/emc/global/noscrub/emc.global/data/ICSDIR/prototype_ICs' PACKAGEROOT: '${PACKAGEROOT:-"/lfs/h1/ops/prod/packages"}' -COMROOT: '${COMROOT:-"/lfs/h1/ops/prod/com"}' -COMINsyn: '${COMROOT}/gfs/v16.3/syndat' +COMINsyn: '/lfs/h1/ops/prod/com/gfs/v16.3/syndat' HOMEDIR: '/lfs/h2/emc/global/noscrub/${USER}' STMP: '/lfs/h2/emc/stmp/${USER}' PTMP: '/lfs/h2/emc/ptmp/${USER}' From db773c937cbac9461bccd3c07eef5efbe8154a92 Mon Sep 17 00:00:00 2001 From: "Kate.Friedman" Date: Thu, 4 Jan 2024 18:59:59 +0000 Subject: [PATCH 03/18] Replace COMROT with ROTDIR in comment Refs #2061 --- scripts/exglobal_forecast.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/exglobal_forecast.sh b/scripts/exglobal_forecast.sh index c6291738798..c50cde74f13 100755 --- a/scripts/exglobal_forecast.sh +++ b/scripts/exglobal_forecast.sh @@ -158,7 +158,7 @@ FV3_out [[ ${cplice} = .true. ]] && CICE_out [[ ${cplchm} = .true. ]] && GOCART_out [[ ${esmf_profile:-} = .true. ]] && CPL_out -echo "MAIN: Output copied to COMROT" +echo "MAIN: Output copied to ROTDIR" #------------------------------------------------------------------ From a7749af56b85fc9e2062565c28a6a1bcca19bc4c Mon Sep 17 00:00:00 2001 From: "Kate.Friedman" Date: Thu, 4 Jan 2024 19:06:43 +0000 Subject: [PATCH 04/18] Replace COMROT/comrot with ROTDIR/comroot in setup - Remove instances of COMROT/comrot in setup scripts - Replace with ROTDIR or comroot as appropriate Refs #2061 --- workflow/setup_expt.py | 64 +++++++++++++++++++++--------------------- 1 file changed, 32 insertions(+), 32 deletions(-) diff --git a/workflow/setup_expt.py b/workflow/setup_expt.py index b1fa4390529..c121c47d68e 100755 --- a/workflow/setup_expt.py +++ b/workflow/setup_expt.py @@ -29,17 +29,17 @@ def makedirs_if_missing(dirname): os.makedirs(dirname) -def fill_COMROT(host, inputs): +def fill_ROTDIR(host, inputs): """ - Method to populate the COMROT for supported modes. + Method to populate the ROTDIR for supported modes. INPUTS: host: host object from class Host inputs: user inputs to setup_expt.py """ fill_modes = { - 'cycled': fill_COMROT_cycled, - 'forecast-only': fill_COMROT_forecasts + 'cycled': fill_ROTDIR_cycled, + 'forecast-only': fill_ROTDIR_forecasts } try: @@ -52,12 +52,12 @@ def fill_COMROT(host, inputs): return -def fill_COMROT_cycled(host, inputs): +def fill_ROTDIR_cycled(host, inputs): """ - Implementation of 'fill_COMROT' for cycled mode + Implementation of 'fill_ROTDIR' for cycled mode """ - comrot = os.path.join(inputs.comrot, inputs.pslot) + rotdir = os.path.join(inputs.comroot, inputs.pslot) do_ocean = do_ice = do_med = False @@ -136,17 +136,17 @@ def link_files_from_src_to_dst(src_dir, dst_dir): memdir = f'mem{ii:03d}' # Link atmospheric files if inputs.start in ['warm']: - dst_dir = os.path.join(comrot, previous_cycle_dir, memdir, dst_atm_dir) + dst_dir = os.path.join(rotdir, previous_cycle_dir, memdir, dst_atm_dir) src_dir = os.path.join(inputs.icsdir, previous_cycle_dir, memdir, src_atm_dir) elif inputs.start in ['cold']: - dst_dir = os.path.join(comrot, current_cycle_dir, memdir, dst_atm_dir) + dst_dir = os.path.join(rotdir, current_cycle_dir, memdir, dst_atm_dir) src_dir = os.path.join(inputs.icsdir, current_cycle_dir, memdir, src_atm_dir) makedirs_if_missing(dst_dir) link_files_from_src_to_dst(src_dir, dst_dir) # Link ocean files if do_ocean: - dst_dir = os.path.join(comrot, previous_cycle_dir, memdir, dst_ocn_rst_dir) + dst_dir = os.path.join(rotdir, previous_cycle_dir, memdir, dst_ocn_rst_dir) src_dir = os.path.join(inputs.icsdir, previous_cycle_dir, memdir, src_ocn_rst_dir) makedirs_if_missing(dst_dir) link_files_from_src_to_dst(src_dir, dst_dir) @@ -154,20 +154,20 @@ def link_files_from_src_to_dst(src_dir, dst_dir): # First 1/2 cycle needs a MOM6 increment incfile = f'enkf{inputs.cdump}.t{idatestr[8:]}z.ocninc.nc' src_file = os.path.join(inputs.icsdir, current_cycle_dir, memdir, src_ocn_anl_dir, incfile) - dst_file = os.path.join(comrot, current_cycle_dir, memdir, dst_ocn_anl_dir, incfile) - makedirs_if_missing(os.path.join(comrot, current_cycle_dir, memdir, dst_ocn_anl_dir)) + dst_file = os.path.join(rotdir, current_cycle_dir, memdir, dst_ocn_anl_dir, incfile) + makedirs_if_missing(os.path.join(rotdir, current_cycle_dir, memdir, dst_ocn_anl_dir)) os.symlink(src_file, dst_file) # Link ice files if do_ice: - dst_dir = os.path.join(comrot, previous_cycle_dir, memdir, dst_ice_rst_dir) + dst_dir = os.path.join(rotdir, previous_cycle_dir, memdir, dst_ice_rst_dir) src_dir = os.path.join(inputs.icsdir, previous_cycle_dir, memdir, src_ice_rst_dir) makedirs_if_missing(dst_dir) link_files_from_src_to_dst(src_dir, dst_dir) # Link mediator files if do_med: - dst_dir = os.path.join(comrot, previous_cycle_dir, memdir, dst_med_dir) + dst_dir = os.path.join(rotdir, previous_cycle_dir, memdir, dst_med_dir) src_dir = os.path.join(inputs.icsdir, previous_cycle_dir, memdir, src_med_dir) makedirs_if_missing(dst_dir) link_files_from_src_to_dst(src_dir, dst_dir) @@ -178,10 +178,10 @@ def link_files_from_src_to_dst(src_dir, dst_dir): # Link atmospheric files if inputs.start in ['warm']: - dst_dir = os.path.join(comrot, previous_cycle_dir, dst_atm_dir) + dst_dir = os.path.join(rotdir, previous_cycle_dir, dst_atm_dir) src_dir = os.path.join(inputs.icsdir, previous_cycle_dir, src_atm_dir) elif inputs.start in ['cold']: - dst_dir = os.path.join(comrot, current_cycle_dir, dst_atm_dir) + dst_dir = os.path.join(rotdir, current_cycle_dir, dst_atm_dir) src_dir = os.path.join(inputs.icsdir, current_cycle_dir, src_atm_dir) makedirs_if_missing(dst_dir) @@ -189,7 +189,7 @@ def link_files_from_src_to_dst(src_dir, dst_dir): # Link ocean files if do_ocean: - dst_dir = os.path.join(comrot, previous_cycle_dir, dst_ocn_rst_dir) + dst_dir = os.path.join(rotdir, previous_cycle_dir, dst_ocn_rst_dir) src_dir = os.path.join(inputs.icsdir, previous_cycle_dir, src_ocn_rst_dir) makedirs_if_missing(dst_dir) link_files_from_src_to_dst(src_dir, dst_dir) @@ -197,27 +197,27 @@ def link_files_from_src_to_dst(src_dir, dst_dir): # First 1/2 cycle needs a MOM6 increment incfile = f'{inputs.cdump}.t{idatestr[8:]}z.ocninc.nc' src_file = os.path.join(inputs.icsdir, current_cycle_dir, src_ocn_anl_dir, incfile) - dst_file = os.path.join(comrot, current_cycle_dir, dst_ocn_anl_dir, incfile) - makedirs_if_missing(os.path.join(comrot, current_cycle_dir, dst_ocn_anl_dir)) + dst_file = os.path.join(rotdir, current_cycle_dir, dst_ocn_anl_dir, incfile) + makedirs_if_missing(os.path.join(rotdir, current_cycle_dir, dst_ocn_anl_dir)) os.symlink(src_file, dst_file) # Link ice files if do_ice: - dst_dir = os.path.join(comrot, previous_cycle_dir, dst_ice_rst_dir) + dst_dir = os.path.join(rotdir, previous_cycle_dir, dst_ice_rst_dir) src_dir = os.path.join(inputs.icsdir, previous_cycle_dir, src_ice_rst_dir) makedirs_if_missing(dst_dir) link_files_from_src_to_dst(src_dir, dst_dir) # Link mediator files if do_med: - dst_dir = os.path.join(comrot, previous_cycle_dir, dst_med_dir) + dst_dir = os.path.join(rotdir, previous_cycle_dir, dst_med_dir) src_dir = os.path.join(inputs.icsdir, previous_cycle_dir, src_med_dir) makedirs_if_missing(dst_dir) link_files_from_src_to_dst(src_dir, dst_dir) # Link bias correction and radiance diagnostics files src_dir = os.path.join(inputs.icsdir, current_cycle_dir, src_atm_anl_dir) - dst_dir = os.path.join(comrot, current_cycle_dir, dst_atm_anl_dir) + dst_dir = os.path.join(rotdir, current_cycle_dir, dst_atm_anl_dir) makedirs_if_missing(dst_dir) for ftype in ['abias', 'abias_pc', 'abias_air', 'radstat']: fname = f'{inputs.cdump}.t{idatestr[8:]}z.{ftype}' @@ -228,9 +228,9 @@ def link_files_from_src_to_dst(src_dir, dst_dir): return -def fill_COMROT_forecasts(host, inputs): +def fill_ROTDIR_forecasts(host, inputs): """ - Implementation of 'fill_COMROT' for forecast-only mode + Implementation of 'fill_ROTDIR' for forecast-only mode """ print('forecast-only mode treats ICs differently and cannot be staged here') @@ -311,7 +311,7 @@ def edit_baseconfig(host, inputs, yaml_dict): "@EDATE@": datetime_to_YMDH(inputs.edate), "@CASECTL@": f'C{inputs.resdet}', "@EXPDIR@": inputs.expdir, - "@ROTDIR@": inputs.comrot, + "@COMROOT@": inputs.comroot, "@EXP_WARM_START@": is_warm_start, "@MODE@": inputs.mode, "@gfs_cyc@": inputs.gfs_cyc, @@ -389,7 +389,7 @@ def _common_args(parser): type=str, required=False, default='test') parser.add_argument('--resdet', help='resolution of the deterministic model forecast', type=int, required=False, default=384) - parser.add_argument('--comrot', help='full path to COMROT', + parser.add_argument('--comroot', help='full path to COMROOT', type=str, required=False, default=os.getenv('HOME')) parser.add_argument('--expdir', help='full path to EXPDIR', type=str, required=False, default=os.getenv('HOME')) @@ -445,7 +445,7 @@ def _gefs_args(parser): description = """ Setup files and directories to start a GFS parallel.\n Create EXPDIR, copy config files.\n - Create COMROT experiment directory structure, + Create ROTDIR experiment directory structure, """ parser = ArgumentParser(description=description, @@ -528,15 +528,15 @@ def main(*argv): validate_user_request(host, user_inputs) - comrot = os.path.join(user_inputs.comrot, user_inputs.pslot) + rotdir = os.path.join(user_inputs.comroot, user_inputs.pslot) expdir = os.path.join(user_inputs.expdir, user_inputs.pslot) - create_comrot = query_and_clean(comrot) + create_rotdir = query_and_clean(rotdir) create_expdir = query_and_clean(expdir) - if create_comrot: - makedirs_if_missing(comrot) - fill_COMROT(host, user_inputs) + if create_rotdir: + makedirs_if_missing(rotdir) + fill_ROTDIR(host, user_inputs) if create_expdir: makedirs_if_missing(expdir) From 48d36aa3d9bf2fac813a8e554f9e78e4b3564fb7 Mon Sep 17 00:00:00 2001 From: "Kate.Friedman" Date: Thu, 4 Jan 2024 19:09:40 +0000 Subject: [PATCH 05/18] Replace rotdir with comroot in README Refs #2061 --- test/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/test/README.md b/test/README.md index 8d9d273ce25..f751d4db5ce 100644 --- a/test/README.md +++ b/test/README.md @@ -24,11 +24,11 @@ Where `dirA` and `dirB` are the two cycle directories (`.../gfs.YYYYMMDD/HH/`) OR ``` -./diff_ROTDIR.sh rotdir cdate expA expB +./diff_ROTDIR.sh comroot cdate expA expB ``` Where: -- `rotdir` is the root of your rotdirs (the portion of path the experiments share) +- `comroot` is the root of your rotdirs (the portion of path the experiments share) - `cdate` is the datetime of the cycle in YYYMMDDHH format - `expA` and `expB` are the experiment names ($PSLOT) of each experiment From 84d46722fd0eb24c192800c84dc8be517601e06b Mon Sep 17 00:00:00 2001 From: "Kate.Friedman" Date: Thu, 4 Jan 2024 19:12:59 +0000 Subject: [PATCH 06/18] Update ROTDIR usage in configs - Replace ROTDIR with COMROOT in base configs for GFS and GEFS - Update function name in config.com Refs #2061 --- parm/config/gefs/config.base.emc.dyn | 2 +- parm/config/gfs/config.base.emc.dyn | 2 +- parm/config/gfs/config.com | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/parm/config/gefs/config.base.emc.dyn b/parm/config/gefs/config.base.emc.dyn index 64a38f669f8..0c54b4b87f3 100644 --- a/parm/config/gefs/config.base.emc.dyn +++ b/parm/config/gefs/config.base.emc.dyn @@ -102,7 +102,7 @@ export EXP_WARM_START="@EXP_WARM_START@" export assim_freq=6 export PSLOT="@PSLOT@" export EXPDIR="@EXPDIR@/${PSLOT}" -export ROTDIR="@ROTDIR@/${PSLOT}" +export ROTDIR="@COMROOT@/${PSLOT}" export DATAROOT="${STMP}/RUNDIRS/${PSLOT}" # TODO: set via prod_envir in Ops export RUNDIR="${DATAROOT}" # TODO: Should be removed; use DATAROOT instead diff --git a/parm/config/gfs/config.base.emc.dyn b/parm/config/gfs/config.base.emc.dyn index 08925c397ee..fc62e16a647 100644 --- a/parm/config/gfs/config.base.emc.dyn +++ b/parm/config/gfs/config.base.emc.dyn @@ -114,7 +114,7 @@ export EXP_WARM_START="@EXP_WARM_START@" export assim_freq=6 export PSLOT="@PSLOT@" export EXPDIR="@EXPDIR@/${PSLOT}" -export ROTDIR="@ROTDIR@/${PSLOT}" +export ROTDIR="@COMROOT@/${PSLOT}" export ROTDIR_DUMP="YES" #Note: A value of "NO" does not currently work export DUMP_SUFFIX="" if [[ "${PDY}${cyc}" -ge "2019092100" && "${PDY}${cyc}" -le "2019110700" ]]; then diff --git a/parm/config/gfs/config.com b/parm/config/gfs/config.com index 208b0ac0968..db648b5866a 100644 --- a/parm/config/gfs/config.com +++ b/parm/config/gfs/config.com @@ -31,7 +31,7 @@ echo "BEGIN: config.com" # # -# If any restart, input, or analysis template is updated, `setup_expt.py.fill_COMROT_cycled()` +# If any restart, input, or analysis template is updated, `setup_expt.py.fill_ROTDIR_cycled()` # must correspondingly be updated to match. # if [[ "${RUN_ENVIR:-emc}" == "nco" ]]; then From 2cab1927c28a2ad28341488d32c4676280488e3a Mon Sep 17 00:00:00 2001 From: "Kate.Friedman" Date: Fri, 5 Jan 2024 13:10:10 +0000 Subject: [PATCH 07/18] Replace COMROT with ROTDIR in ci system Refs #2061 --- ci/cases/pr/C48_ATM.yaml | 2 +- ci/cases/pr/C48_S2SW.yaml | 2 +- ci/cases/pr/C48_S2SWA_gefs.yaml | 2 +- ci/cases/pr/C96C48_hybatmDA.yaml | 2 +- ci/cases/pr/C96_atm3DVar.yaml | 2 +- ci/cases/weekly/C384C192_hybatmda.yaml | 2 +- ci/cases/weekly/C384_S2SWA.yaml | 2 +- ci/cases/weekly/C384_atm3DVar.yaml | 2 +- ci/scripts/check_ci.sh | 2 +- ci/scripts/run-check_ci.sh | 4 ++-- 10 files changed, 11 insertions(+), 11 deletions(-) diff --git a/ci/cases/pr/C48_ATM.yaml b/ci/cases/pr/C48_ATM.yaml index fc0b729af65..ffd37409081 100644 --- a/ci/cases/pr/C48_ATM.yaml +++ b/ci/cases/pr/C48_ATM.yaml @@ -6,7 +6,7 @@ arguments: pslot: {{ 'pslot' | getenv }} app: ATM resdet: 48 - comrot: {{ 'RUNTESTS' | getenv }}/COMROT + rotdir: {{ 'RUNTESTS' | getenv }}/ROTDIR expdir: {{ 'RUNTESTS' | getenv }}/EXPDIR idate: 2021032312 edate: 2021032312 diff --git a/ci/cases/pr/C48_S2SW.yaml b/ci/cases/pr/C48_S2SW.yaml index f4b50ead222..d7f781f1180 100644 --- a/ci/cases/pr/C48_S2SW.yaml +++ b/ci/cases/pr/C48_S2SW.yaml @@ -6,7 +6,7 @@ arguments: pslot: {{ 'pslot' | getenv }} app: S2SW resdet: 48 - comrot: {{ 'RUNTESTS' | getenv }}/COMROT + rotdir: {{ 'RUNTESTS' | getenv }}/ROTDIR expdir: {{ 'RUNTESTS' | getenv }}/EXPDIR idate: 2021032312 edate: 2021032312 diff --git a/ci/cases/pr/C48_S2SWA_gefs.yaml b/ci/cases/pr/C48_S2SWA_gefs.yaml index 5eb99d9c1ef..0f6fc84fd4a 100644 --- a/ci/cases/pr/C48_S2SWA_gefs.yaml +++ b/ci/cases/pr/C48_S2SWA_gefs.yaml @@ -10,7 +10,7 @@ arguments: nens: 2 gfs_cyc: 1 start: cold - comrot: {{ 'RUNTESTS' | getenv }}/COMROT + rotdir: {{ 'RUNTESTS' | getenv }}/ROTDIR expdir: {{ 'RUNTESTS' | getenv }}/EXPDIR idate: 2021032312 edate: 2021032312 diff --git a/ci/cases/pr/C96C48_hybatmDA.yaml b/ci/cases/pr/C96C48_hybatmDA.yaml index 1f3e973ae73..69a8eed6576 100644 --- a/ci/cases/pr/C96C48_hybatmDA.yaml +++ b/ci/cases/pr/C96C48_hybatmDA.yaml @@ -7,7 +7,7 @@ arguments: app: ATM resdet: 96 resens: 48 - comrot: {{ 'RUNTESTS' | getenv }}/COMROT + rotdir: {{ 'RUNTESTS' | getenv }}/ROTDIR expdir: {{ 'RUNTESTS' | getenv }}/EXPDIR icsdir: {{ 'ICSDIR_ROOT' | getenv }}/C96C48 idate: 2021122018 diff --git a/ci/cases/pr/C96_atm3DVar.yaml b/ci/cases/pr/C96_atm3DVar.yaml index 360e81e9d76..ea07f96e30b 100644 --- a/ci/cases/pr/C96_atm3DVar.yaml +++ b/ci/cases/pr/C96_atm3DVar.yaml @@ -6,7 +6,7 @@ arguments: pslot: {{ 'pslot' | getenv }} app: ATM resdet: 96 - comrot: {{ 'RUNTESTS' | getenv }}/COMROT + rotdir: {{ 'RUNTESTS' | getenv }}/ROTDIR expdir: {{ 'RUNTESTS' | getenv }}/EXPDIR icsdir: ${ICSDIR_ROOT}/C96C48 idate: 2021122018 diff --git a/ci/cases/weekly/C384C192_hybatmda.yaml b/ci/cases/weekly/C384C192_hybatmda.yaml index 4c14018e2da..c7a9db6fd05 100644 --- a/ci/cases/weekly/C384C192_hybatmda.yaml +++ b/ci/cases/weekly/C384C192_hybatmda.yaml @@ -7,7 +7,7 @@ arguments: app: ATM resdet: 384 resens: 192 - comrot: {{ 'RUNTESTS' | getenv }}/COMROT + rotdir: {{ 'RUNTESTS' | getenv }}/ROTDIR expdir: {{ 'RUNTESTS' | getenv }}/EXPDIR icsdir: {{ 'ICSDIR_ROOT' | getenv }}/C384C192 idate: 2023040118 diff --git a/ci/cases/weekly/C384_S2SWA.yaml b/ci/cases/weekly/C384_S2SWA.yaml index 6c624f56983..573295a4522 100644 --- a/ci/cases/weekly/C384_S2SWA.yaml +++ b/ci/cases/weekly/C384_S2SWA.yaml @@ -6,7 +6,7 @@ arguments: pslot: {{ 'pslot' | getenv }} app: S2SWA resdet: 384 - comrot: {{ 'RUNTESTS' | getenv }}/COMROT + rotdir: {{ 'RUNTESTS' | getenv }}/ROTDIR expdir: {{ 'RUNTESTS' | getenv }}/EXPDIR idate: 2016070100 edate: 2016070100 diff --git a/ci/cases/weekly/C384_atm3DVar.yaml b/ci/cases/weekly/C384_atm3DVar.yaml index e7986ef0976..1bec5041938 100644 --- a/ci/cases/weekly/C384_atm3DVar.yaml +++ b/ci/cases/weekly/C384_atm3DVar.yaml @@ -7,7 +7,7 @@ arguments: app: ATM resdet: 384 resens: 192 - comrot: {{ 'RUNTESTS' | getenv }}/COMROT + rotdir: {{ 'RUNTESTS' | getenv }}/ROTDIR expdir: {{ 'RUNTESTS' | getenv }}/EXPDIR icsdir: {{ 'ICSDIR_ROOT' | getenv }}/C384C192 idate: 2023040118 diff --git a/ci/scripts/check_ci.sh b/ci/scripts/check_ci.sh index 164d423c674..30be22ac178 100755 --- a/ci/scripts/check_ci.sh +++ b/ci/scripts/check_ci.sh @@ -149,7 +149,7 @@ for pr in ${pr_list}; do if [[ "${num_done}" -eq "${num_cycles}" ]]; then #Remove Experment cases that completed successfully rm -Rf "${pslot_dir}" - rm -Rf "${pr_dir}/RUNTESTS/COMROT/${pslot}" + rm -Rf "${pr_dir}/RUNTESTS/ROTDIR/${pslot}" rm -f "${output_ci_single}" # echo "\`\`\`" > "${output_ci_single}" DATE=$(date +'%D %r') diff --git a/ci/scripts/run-check_ci.sh b/ci/scripts/run-check_ci.sh index a5c5369ef7a..00130d522ef 100755 --- a/ci/scripts/run-check_ci.sh +++ b/ci/scripts/run-check_ci.sh @@ -12,12 +12,12 @@ pslot=${2:-${pslot:-?}} # Name of the experiment being tested by this scr # TEST_DIR contains 2 directories; # 1. HOMEgfs: clone of the global-workflow -# 2. RUNTESTS: A directory containing EXPDIR and COMROT for experiments +# 2. RUNTESTS: A directory containing EXPDIR and ROTDIR for experiments # # e.g. $> tree ./TEST_DIR # ./TEST_DIR # ├── HOMEgfs # └── RUNTESTS -# ├── COMROT +# ├── ROTDIR # │   └── ${pslot} # └── EXPDIR # └── ${pslot} From 2e5fa57ae634d64c1b09bc73bdb1af4fb299dd32 Mon Sep 17 00:00:00 2001 From: Rahul Mahajan Date: Sun, 7 Jan 2024 23:20:43 -0500 Subject: [PATCH 08/18] add option to provide an ocean resolution during setup_expt.py --- parm/config/gefs/config.base.emc.dyn | 24 ++++++++--------------- parm/config/gfs/config.base.emc.dyn | 16 ++++----------- workflow/setup_expt.py | 29 ++++++++++++++++++++++++---- 3 files changed, 37 insertions(+), 32 deletions(-) diff --git a/parm/config/gefs/config.base.emc.dyn b/parm/config/gefs/config.base.emc.dyn index 64a38f669f8..e1977197e3c 100644 --- a/parm/config/gefs/config.base.emc.dyn +++ b/parm/config/gefs/config.base.emc.dyn @@ -154,35 +154,27 @@ export OPS_RES="C768" # Do not change export LEVS=128 export CASE="@CASECTL@" # CASE is required in GEFS to determine ocean/ice/wave resolutions export CASE_ENS="@CASEENS@" -# TODO: This should not depend on $CASE or $CASE_ENS -# These are the currently available grid-combinations +export OCNRES="@OCNRES@" +export ICERES="${OCNRES}" +# These are the currently recommended grid-combinations case "${CASE}" in "C48") - export OCNRES=500 export waveGRD='glo_500' ;; - "C96") - export OCNRES=100 - export waveGRD='glo_200' - ;; - "C192") - export OCNRES=050 + "C96" | "C192") export waveGRD='glo_200' ;; "C384") - export OCNRES=025 export waveGRD='glo_025' ;; - "C768") - export OCNRES=025 - export waveGRD='mx025' + "C768" | "C1152") + export waveGRD='mx025' ;; *) - export OCNRES=025 - export waveGRD='glo_025' + echo "FATAL ERROR: Unrecognized CASE ${CASE}, ABORT!" + exit 1 ;; esac -export ICERES=${OCNRES} case "${APP}" in ATM) diff --git a/parm/config/gfs/config.base.emc.dyn b/parm/config/gfs/config.base.emc.dyn index 08925c397ee..95a22369f32 100644 --- a/parm/config/gfs/config.base.emc.dyn +++ b/parm/config/gfs/config.base.emc.dyn @@ -170,27 +170,20 @@ export OPS_RES="C768" # Do not change # TODO: Why is this needed and where is it export LEVS=128 export CASE="@CASECTL@" export CASE_ENS="@CASEENS@" -# TODO: This should not depend on $CASE or $CASE_ENS -# These are the currently available grid-combinations +export OCNRES="@OCNRES@" +export ICERES="${OCNRES}" +# These are the currently recommended grid-combinations case "${CASE}" in "C48") - export OCNRES=500 export waveGRD='glo_500' ;; - "C96") - export OCNRES=500 - export waveGRD='glo_200' - ;; - "C192") - export OCNRES=050 + "C96" | "C192") export waveGRD='glo_200' ;; "C384") - export OCNRES=025 export waveGRD='glo_025' ;; "C768" | "C1152") - export OCNRES=025 export waveGRD='mx025' ;; *) @@ -198,7 +191,6 @@ case "${CASE}" in exit 1 ;; esac -export ICERES=${OCNRES} case "${APP}" in ATM) diff --git a/workflow/setup_expt.py b/workflow/setup_expt.py index b1fa4390529..f254a0e3375 100755 --- a/workflow/setup_expt.py +++ b/workflow/setup_expt.py @@ -309,7 +309,8 @@ def edit_baseconfig(host, inputs, yaml_dict): "@PSLOT@": inputs.pslot, "@SDATE@": datetime_to_YMDH(inputs.idate), "@EDATE@": datetime_to_YMDH(inputs.edate), - "@CASECTL@": f'C{inputs.resdet}', + "@CASECTL@": f'C{inputs.resdetatm}', + "@OCNRES@": f"{int(100.*inputs.resdetocean):03d}", "@EXPDIR@": inputs.expdir, "@ROTDIR@": inputs.comrot, "@EXP_WARM_START@": is_warm_start, @@ -387,8 +388,10 @@ def input_args(*argv): def _common_args(parser): parser.add_argument('--pslot', help='parallel experiment name', type=str, required=False, default='test') - parser.add_argument('--resdet', help='resolution of the deterministic model forecast', + parser.add_argument('--resdetatm', help='atmosphere resolution of the deterministic model forecast', type=int, required=False, default=384) + parser.add_argument('--resdetocean', help='ocean resolution of the deterministic model forecast', + type=float, required=False, default=-1.0) # -1.0 means determine from resdetatm (limited combinations will be available) parser.add_argument('--comrot', help='full path to COMROT', type=str, required=False, default=os.getenv('HOME')) parser.add_argument('--expdir', help='full path to EXPDIR', @@ -418,7 +421,7 @@ def _gfs_cycled_args(parser): return parser def _gfs_or_gefs_ensemble_args(parser): - parser.add_argument('--resens', help='resolution of the ensemble model forecast', + parser.add_argument('--resensatm', help='atmosphere esolution of the ensemble model forecast', type=int, required=False, default=192) parser.add_argument('--nens', help='number of ensemble members', type=int, required=False, default=20) @@ -512,7 +515,7 @@ def query_and_clean(dirname): def validate_user_request(host, inputs): supp_res = host.info['SUPPORTED_RESOLUTIONS'] machine = host.machine - for attr in ['resdet', 'resens']: + for attr in ['resdetatm', 'resensatm']: try: expt_res = f'C{getattr(inputs, attr)}' except AttributeError: @@ -521,6 +524,20 @@ def validate_user_request(host, inputs): raise NotImplementedError(f"Supported resolutions on {machine} are:\n{', '.join(supp_res)}") +def get_ocean_resolution(resdetatm): + """ + Method to determine the ocean resolution based on the atmosphere resolution + Limited options are going to be available + """ + atm_to_ocean_map = { + 1152: 0.25, 768: 0.25, 384: 0.25, + 192: 1.0, + 96: 5.0, 48: 5.0} + try: + return atm_to_ocean_map[resdetatm] + except KeyError: + raise KeyError(f"Ocean resolution for {resdetatm} is not implemented") + def main(*argv): user_inputs = input_args(*argv) @@ -528,6 +545,10 @@ def main(*argv): validate_user_request(host, user_inputs) + # Determine ocean resolution if not provided + if user_inputs.resdetocean < 0: + user_inputs.resdetocean = get_ocean_resolution(user_inputs.resdetatm) + comrot = os.path.join(user_inputs.comrot, user_inputs.pslot) expdir = os.path.join(user_inputs.expdir, user_inputs.pslot) From 740ef66c4dac7d038bc8c2f7eb4c963468c897bb Mon Sep 17 00:00:00 2001 From: Rahul Mahajan Date: Mon, 8 Jan 2024 10:00:42 -0500 Subject: [PATCH 09/18] fix pynorm error --- workflow/setup_expt.py | 1 + 1 file changed, 1 insertion(+) diff --git a/workflow/setup_expt.py b/workflow/setup_expt.py index f254a0e3375..35b8e8276d9 100755 --- a/workflow/setup_expt.py +++ b/workflow/setup_expt.py @@ -538,6 +538,7 @@ def get_ocean_resolution(resdetatm): except KeyError: raise KeyError(f"Ocean resolution for {resdetatm} is not implemented") + def main(*argv): user_inputs = input_args(*argv) From 52876b91e1c59f4a5b3a33b6c4d75413ae90bfd3 Mon Sep 17 00:00:00 2001 From: Rahul Mahajan Date: Tue, 9 Jan 2024 11:23:28 -0500 Subject: [PATCH 10/18] update yamls for ci tests --- ci/cases/pr/C48_ATM.yaml | 2 +- ci/cases/pr/C48_S2SW.yaml | 3 ++- ci/cases/pr/C48_S2SWA_gefs.yaml | 5 +++-- ci/cases/pr/C96C48_hybatmDA.yaml | 5 +++-- ci/cases/pr/C96_atm3DVar.yaml | 4 ++-- ci/cases/weekly/C384C192_hybatmda.yaml | 5 +++-- ci/cases/weekly/C384_S2SWA.yaml | 3 ++- ci/cases/weekly/C384_atm3DVar.yaml | 5 +++-- ci/platforms/gefs_ci_defaults.yaml | 2 +- ci/platforms/gfs_defaults_ci.yaml | 2 +- workflow/setup_expt.py | 22 +++++++++++----------- 11 files changed, 32 insertions(+), 26 deletions(-) diff --git a/ci/cases/pr/C48_ATM.yaml b/ci/cases/pr/C48_ATM.yaml index fc0b729af65..dfe0c51ba2e 100644 --- a/ci/cases/pr/C48_ATM.yaml +++ b/ci/cases/pr/C48_ATM.yaml @@ -5,7 +5,7 @@ experiment: arguments: pslot: {{ 'pslot' | getenv }} app: ATM - resdet: 48 + resdetatmos: 48 comrot: {{ 'RUNTESTS' | getenv }}/COMROT expdir: {{ 'RUNTESTS' | getenv }}/EXPDIR idate: 2021032312 diff --git a/ci/cases/pr/C48_S2SW.yaml b/ci/cases/pr/C48_S2SW.yaml index f4b50ead222..d92159efbee 100644 --- a/ci/cases/pr/C48_S2SW.yaml +++ b/ci/cases/pr/C48_S2SW.yaml @@ -5,7 +5,8 @@ experiment: arguments: pslot: {{ 'pslot' | getenv }} app: S2SW - resdet: 48 + resdetatmos: 48 + resdetocean: 5.0 comrot: {{ 'RUNTESTS' | getenv }}/COMROT expdir: {{ 'RUNTESTS' | getenv }}/EXPDIR idate: 2021032312 diff --git a/ci/cases/pr/C48_S2SWA_gefs.yaml b/ci/cases/pr/C48_S2SWA_gefs.yaml index 5eb99d9c1ef..b4bde62feb7 100644 --- a/ci/cases/pr/C48_S2SWA_gefs.yaml +++ b/ci/cases/pr/C48_S2SWA_gefs.yaml @@ -5,8 +5,9 @@ experiment: arguments: pslot: {{ 'pslot' | getenv }} app: S2SWA - resdet: 48 - resens: 48 + resdetatmos: 48 + resdetocean: 5.0 + resensatmos: 48 nens: 2 gfs_cyc: 1 start: cold diff --git a/ci/cases/pr/C96C48_hybatmDA.yaml b/ci/cases/pr/C96C48_hybatmDA.yaml index c3aa6e88921..7b074b883e3 100644 --- a/ci/cases/pr/C96C48_hybatmDA.yaml +++ b/ci/cases/pr/C96C48_hybatmDA.yaml @@ -5,8 +5,9 @@ experiment: arguments: pslot: {{ 'pslot' | getenv }} app: ATM - resdet: 96 - resens: 48 + resdetatmos: 96 + resdetocean: 5.0 + resensatmos: 48 comrot: {{ 'RUNTESTS' | getenv }}/COMROT expdir: {{ 'RUNTESTS' | getenv }}/EXPDIR icsdir: {{ 'ICSDIR_ROOT' | getenv }}/C96C48 diff --git a/ci/cases/pr/C96_atm3DVar.yaml b/ci/cases/pr/C96_atm3DVar.yaml index 5215cb0d903..b068ebda27d 100644 --- a/ci/cases/pr/C96_atm3DVar.yaml +++ b/ci/cases/pr/C96_atm3DVar.yaml @@ -5,10 +5,10 @@ experiment: arguments: pslot: {{ 'pslot' | getenv }} app: ATM - resdet: 96 + resdetatmos: 96 comrot: {{ 'RUNTESTS' | getenv }}/COMROT expdir: {{ 'RUNTESTS' | getenv }}/EXPDIR - icsdir: ${ICSDIR_ROOT}/C96C48 + icsdir: {{ 'ICSDIR_ROOT' | getenv ]}/C96C48 idate: 2021122018 edate: 2021122106 nens: 0 diff --git a/ci/cases/weekly/C384C192_hybatmda.yaml b/ci/cases/weekly/C384C192_hybatmda.yaml index 4c14018e2da..3102ec850b0 100644 --- a/ci/cases/weekly/C384C192_hybatmda.yaml +++ b/ci/cases/weekly/C384C192_hybatmda.yaml @@ -5,8 +5,9 @@ experiment: arguments: pslot: {{ 'pslot' | getenv }} app: ATM - resdet: 384 - resens: 192 + resdetatmos: 384 + resdetocean: 0.25 + resensatmos: 192 comrot: {{ 'RUNTESTS' | getenv }}/COMROT expdir: {{ 'RUNTESTS' | getenv }}/EXPDIR icsdir: {{ 'ICSDIR_ROOT' | getenv }}/C384C192 diff --git a/ci/cases/weekly/C384_S2SWA.yaml b/ci/cases/weekly/C384_S2SWA.yaml index 6c624f56983..a80263fac05 100644 --- a/ci/cases/weekly/C384_S2SWA.yaml +++ b/ci/cases/weekly/C384_S2SWA.yaml @@ -5,7 +5,8 @@ experiment: arguments: pslot: {{ 'pslot' | getenv }} app: S2SWA - resdet: 384 + resdetatmos: 384 + resdetocean: 0.5 comrot: {{ 'RUNTESTS' | getenv }}/COMROT expdir: {{ 'RUNTESTS' | getenv }}/EXPDIR idate: 2016070100 diff --git a/ci/cases/weekly/C384_atm3DVar.yaml b/ci/cases/weekly/C384_atm3DVar.yaml index e7986ef0976..194d66c8ceb 100644 --- a/ci/cases/weekly/C384_atm3DVar.yaml +++ b/ci/cases/weekly/C384_atm3DVar.yaml @@ -5,8 +5,9 @@ experiment: arguments: pslot: {{ 'pslot' | getenv }} app: ATM - resdet: 384 - resens: 192 + resdetatmos: 384 + resdetocean: 0.25 + resensatmos: 192 comrot: {{ 'RUNTESTS' | getenv }}/COMROT expdir: {{ 'RUNTESTS' | getenv }}/EXPDIR icsdir: {{ 'ICSDIR_ROOT' | getenv }}/C384C192 diff --git a/ci/platforms/gefs_ci_defaults.yaml b/ci/platforms/gefs_ci_defaults.yaml index 2aa30d6be44..dfb1626cdd8 100644 --- a/ci/platforms/gefs_ci_defaults.yaml +++ b/ci/platforms/gefs_ci_defaults.yaml @@ -1,4 +1,4 @@ defaults: !INC {{ HOMEgfs }}/parm/config/gefs/yaml/defaults.yaml base: - ACCOUNT: ${SLURM_ACCOUNT} + ACCOUNT: {{ 'SLURM_ACCOUNT' | getenv }} diff --git a/ci/platforms/gfs_defaults_ci.yaml b/ci/platforms/gfs_defaults_ci.yaml index 5e57e617ec7..b66be2a366f 100644 --- a/ci/platforms/gfs_defaults_ci.yaml +++ b/ci/platforms/gfs_defaults_ci.yaml @@ -1,4 +1,4 @@ defaults: !INC {{ HOMEgfs }}/parm/config/gfs/yaml/defaults.yaml base: - ACCOUNT: ${SLURM_ACCOUNT} + ACCOUNT: {{ 'SLURM_ACCOUNT' | getenv }} diff --git a/workflow/setup_expt.py b/workflow/setup_expt.py index 35b8e8276d9..fe94c9ca3d7 100755 --- a/workflow/setup_expt.py +++ b/workflow/setup_expt.py @@ -309,7 +309,7 @@ def edit_baseconfig(host, inputs, yaml_dict): "@PSLOT@": inputs.pslot, "@SDATE@": datetime_to_YMDH(inputs.idate), "@EDATE@": datetime_to_YMDH(inputs.edate), - "@CASECTL@": f'C{inputs.resdetatm}', + "@CASECTL@": f'C{inputs.resdetatmos}', "@OCNRES@": f"{int(100.*inputs.resdetocean):03d}", "@EXPDIR@": inputs.expdir, "@ROTDIR@": inputs.comrot, @@ -323,7 +323,7 @@ def edit_baseconfig(host, inputs, yaml_dict): extend_dict = dict() if getattr(inputs, 'nens', 0) > 0: extend_dict = { - "@CASEENS@": f'C{inputs.resens}', + "@CASEENS@": f'C{inputs.resensatmos}', "@NMEM_ENS@": inputs.nens, } tmpl_dict = dict(tmpl_dict, **extend_dict) @@ -388,10 +388,10 @@ def input_args(*argv): def _common_args(parser): parser.add_argument('--pslot', help='parallel experiment name', type=str, required=False, default='test') - parser.add_argument('--resdetatm', help='atmosphere resolution of the deterministic model forecast', + parser.add_argument('--resdetatmos', help='atmosphere resolution of the deterministic model forecast', type=int, required=False, default=384) parser.add_argument('--resdetocean', help='ocean resolution of the deterministic model forecast', - type=float, required=False, default=-1.0) # -1.0 means determine from resdetatm (limited combinations will be available) + type=float, required=False, default=-1.0) # -1.0 means determine from resdetatmos (limited combinations will be available) parser.add_argument('--comrot', help='full path to COMROT', type=str, required=False, default=os.getenv('HOME')) parser.add_argument('--expdir', help='full path to EXPDIR', @@ -421,7 +421,7 @@ def _gfs_cycled_args(parser): return parser def _gfs_or_gefs_ensemble_args(parser): - parser.add_argument('--resensatm', help='atmosphere esolution of the ensemble model forecast', + parser.add_argument('--resensatmos', help='atmosphere resolution of the ensemble model forecast', type=int, required=False, default=192) parser.add_argument('--nens', help='number of ensemble members', type=int, required=False, default=20) @@ -515,7 +515,7 @@ def query_and_clean(dirname): def validate_user_request(host, inputs): supp_res = host.info['SUPPORTED_RESOLUTIONS'] machine = host.machine - for attr in ['resdetatm', 'resensatm']: + for attr in ['resdetatmos', 'resensatmos']: try: expt_res = f'C{getattr(inputs, attr)}' except AttributeError: @@ -524,19 +524,19 @@ def validate_user_request(host, inputs): raise NotImplementedError(f"Supported resolutions on {machine} are:\n{', '.join(supp_res)}") -def get_ocean_resolution(resdetatm): +def get_ocean_resolution(resdetatmos): """ Method to determine the ocean resolution based on the atmosphere resolution Limited options are going to be available """ - atm_to_ocean_map = { + atmos_to_ocean_map = { 1152: 0.25, 768: 0.25, 384: 0.25, 192: 1.0, 96: 5.0, 48: 5.0} try: - return atm_to_ocean_map[resdetatm] + return atmos_to_ocean_map[resdetatmos] except KeyError: - raise KeyError(f"Ocean resolution for {resdetatm} is not implemented") + raise KeyError(f"Ocean resolution for {resdetatmos} is not implemented") def main(*argv): @@ -548,7 +548,7 @@ def main(*argv): # Determine ocean resolution if not provided if user_inputs.resdetocean < 0: - user_inputs.resdetocean = get_ocean_resolution(user_inputs.resdetatm) + user_inputs.resdetocean = get_ocean_resolution(user_inputs.resdetatmos) comrot = os.path.join(user_inputs.comrot, user_inputs.pslot) expdir = os.path.join(user_inputs.expdir, user_inputs.pslot) From f8da86558e1bda763e1eda8937ad7d2b1b341f3a Mon Sep 17 00:00:00 2001 From: Rahul Mahajan Date: Tue, 9 Jan 2024 11:42:04 -0500 Subject: [PATCH 11/18] update documentation instances of resdet and resens; add resdetocean --- docs/doxygen/mainpage.h | 10 ++++++---- docs/source/setup.rst | 26 ++++++++++++++------------ 2 files changed, 20 insertions(+), 16 deletions(-) diff --git a/docs/doxygen/mainpage.h b/docs/doxygen/mainpage.h index 40e8e6f946b..5d8ca875503 100644 --- a/docs/doxygen/mainpage.h +++ b/docs/doxygen/mainpage.h @@ -23,7 +23,7 @@ To setup an experiment, a python script \c setup_expt.py (located in \ $> setup_expt.py -h usage: setup_expt.py [-h] --pslot PSLOT [--configdir CONFIGDIR] [--idate IDATE] [--icsdir ICSDIR] - [--resdet RESDET] [--resens RESENS] [--comrot COMROT] + [--resdetatmos RESDET] [--resensatmos RESENS] [--comrot COMROT] [--expdir EXPDIR] [--nens NENS] [--cdump CDUMP] Setup files and directories to start a GFS parallel. Create EXPDIR, copy @@ -40,9 +40,11 @@ To setup an experiment, a python script \c setup_expt.py (located in \ (default: 2016100100) --icsdir full path to initial condition directory (default: /scratch4/NCEPDEV/da/noscrub/Rahul.Mahajan/ICS) - --resdet resolution of the deterministic model forecast + --resdetatmos atmosphere resolution of the deterministic model forecast (default: 384) - --resens resolution of the ensemble model forecast + --resdetocean ocean resolution of the deterministic model forecast + (default: 0. [determined automatically based on atmosphere resolution]]]) + --resensatmos resolution of the ensemble model forecast (default: 192) --comrot full path to COMROT (default: None) @@ -53,7 +55,7 @@ To setup an experiment, a python script \c setup_expt.py (located in \ --cdump CDUMP to start the experiment (default: gdas) -The above script creates directories \c EXPDIR and \c COMROT. It will make links for initial conditions from a location provided via the \c --icsdir argument for a chosen resolution for the control \c --resdet and the ensemble \c --resens. Experiment name is controlled by the input argument \c --pslot. The script will ask user input in case any of the directories already exist. It will copy experiment configuration files into the \c EXPDIR from \c CONFIGDIR. +The above script creates directories \c EXPDIR and \c COMROT. It will make links for initial conditions from a location provided via the \c --icsdir argument for a chosen resolution for the control \c --resdetatmos and the ensemble \c --resensatmos. Experiment name is controlled by the input argument \c --pslot. The script will ask user input in case any of the directories already exist. It will copy experiment configuration files into the \c EXPDIR from \c CONFIGDIR. Sample initial conditions for a few resolutions are available at:
Hera: TODO: /path/here/for/initial/conditions
diff --git a/docs/source/setup.rst b/docs/source/setup.rst index be04aa5d96c..eaf42b18e0d 100644 --- a/docs/source/setup.rst +++ b/docs/source/setup.rst @@ -28,7 +28,7 @@ The following command examples include variables for reference but users should :: cd workflow - ./setup_expt.py gfs forecast-only --idate $IDATE --edate $EDATE [--app $APP] [--start $START] [--gfs_cyc $GFS_CYC] [--resdet $RESDET] + ./setup_expt.py gfs forecast-only --idate $IDATE --edate $EDATE [--app $APP] [--start $START] [--gfs_cyc $GFS_CYC] [--resdetatmos $RESDETATMOS] [--resdetocean $RESDETOCEAN] [--pslot $PSLOT] [--configdir $CONFIGDIR] [--comrot $COMROT] [--expdir $EXPDIR] where: @@ -50,7 +50,8 @@ where: * ``$EDATE`` is the ending date of your run (YYYYMMDDCC) and is the last cycle that will complete * ``$PSLOT`` is the name of your experiment [default: test] * ``$CONFIGDIR`` is the path to the ``/config`` folder under the copy of the system you're using [default: $TOP_OF_CLONE/parm/config/] - * ``$RESDET`` is the FV3 resolution (i.e. 768 for C768) [default: 384] + * ``$RESDETATMOS`` is the resolution of the atmosphere component of the system (i.e. 768 for C768) [default: 384] + * ``$RESDETOCEAN`` is the resolution of the ocean component of the system (i.e. 0.25 for 1/4 degree) [default: 0.; determined based on atmosphere resolution] * ``$GFS_CYC`` is the forecast frequency (0 = none, 1 = 00z only [default], 2 = 00z & 12z, 4 = all cycles) * ``$COMROT`` is the path to your experiment output directory. DO NOT include PSLOT folder at end of path, it’ll be built for you. [default: $HOME (but do not use default due to limited space in home directories normally, provide a path to a larger scratch space)] * ``$EXPDIR`` is the path to your experiment directory where your configs will be placed and where you will find your workflow monitoring files (i.e. rocoto database and xml file). DO NOT include PSLOT folder at end of path, it will be built for you. [default: $HOME] @@ -62,21 +63,21 @@ Atm-only: :: cd workflow - ./setup_expt.py gfs forecast-only --pslot test --idate 2020010100 --edate 2020010118 --resdet 384 --gfs_cyc 4 --comrot /some_large_disk_area/Joe.Schmo/comrot --expdir /some_safe_disk_area/Joe.Schmo/expdir + ./setup_expt.py gfs forecast-only --pslot test --idate 2020010100 --edate 2020010118 --resdetatmos 384 --gfs_cyc 4 --comrot /some_large_disk_area/Joe.Schmo/comrot --expdir /some_safe_disk_area/Joe.Schmo/expdir Coupled: :: cd workflow - ./setup_expt.py gfs forecast-only --app S2SW --pslot coupled_test --idate 2013040100 --edate 2013040100 --resdet 384 --comrot /some_large_disk_area/Joe.Schmo/comrot --expdir /some_safe_disk_area/Joe.Schmo/expdir + ./setup_expt.py gfs forecast-only --app S2SW --pslot coupled_test --idate 2013040100 --edate 2013040100 --resdetatmos 384 --comrot /some_large_disk_area/Joe.Schmo/comrot --expdir /some_safe_disk_area/Joe.Schmo/expdir Coupled with aerosols: :: cd workflow - ./setup_expt.py gfs forecast-only --app S2SWA --pslot coupled_test --idate 2013040100 --edate 2013040100 --resdet 384 --comrot /some_large_disk_area/Joe.Schmo/comrot --expdir /some_safe_disk_area/Joe.Schmo/expdir + ./setup_expt.py gfs forecast-only --app S2SWA --pslot coupled_test --idate 2013040100 --edate 2013040100 --resdetatmos 384 --comrot /some_large_disk_area/Joe.Schmo/comrot --expdir /some_safe_disk_area/Joe.Schmo/expdir **************************************** Step 2: Set user and experiment settings @@ -134,13 +135,13 @@ Scripts that will be used: Step 1) Run experiment generator script *************************************** -The following command examples include variables for reference but users should not use environmental variables but explicit values to submit the commands. Exporting variables like EXPDIR to your environment causes an error when the python scripts run. Please explicitly include the argument inputs when running both setup scripts: +The following command examples include variables for reference but users should not use environment variables but explicit values to submit the commands. Exporting variables like EXPDIR to your environment causes an error when the python scripts run. Please explicitly include the argument inputs when running both setup scripts: :: cd workflow ./setup_expt.py gfs cycled --idate $IDATE --edate $EDATE [--app $APP] [--start $START] [--gfs_cyc $GFS_CYC] - [--resdet $RESDET] [--resens $RESENS] [--nens $NENS] [--cdump $CDUMP] + [--resdetatmos $RESDETATMOS] [--resdetocean $RESDETOCEAN] [--resensatmos $RESENSATMOS] [--nens $NENS] [--cdump $CDUMP] [--pslot $PSLOT] [--configdir $CONFIGDIR] [--comrot $COMROT] [--expdir $EXPDIR] [--icsdir $ICSDIR] where: @@ -161,8 +162,9 @@ where: * ``$EDATE`` is the ending date of your run (YYYYMMDDCC) and is the last cycle that will complete * ``$START`` is the start type (warm or cold [default]) * ``$GFS_CYC`` is the forecast frequency (0 = none, 1 = 00z only [default], 2 = 00z & 12z, 4 = all cycles) - * ``$RESDET`` is the FV3 resolution of the deterministic forecast [default: 384] - * ``$RESENS`` is the FV3 resolution of the ensemble (EnKF) forecast [default: 192] + * ``$RESDETATMOS`` is the resolution of the atmosphere component of the deterministic forecast [default: 384] + * ``$RESDETOCEAN`` is the resolution of the ocean component of the deterministic forecast [default: 0.; determined based on atmosphere resolution] + * ``$RESENSATMOS`` is the resolution of the atmosphere component of the ensemble forecast [default: 192] * ``$NENS`` is the number of ensemble members [default: 20] * ``$CDUMP`` is the starting phase [default: gdas] * ``$PSLOT`` is the name of your experiment [default: test] @@ -178,13 +180,13 @@ Example: :: cd workflow - ./setup_expt.py gfs cycled --pslot test --configdir /home/Joe.Schmo/git/global-workflow/parm/config --idate 2020010100 --edate 2020010118 --comrot /some_large_disk_area/Joe.Schmo/comrot --expdir /some_safe_disk_area/Joe.Schmo/expdir --resdet 384 --resens 192 --nens 80 --gfs_cyc 4 + ./setup_expt.py gfs cycled --pslot test --configdir /home/Joe.Schmo/git/global-workflow/parm/config --idate 2020010100 --edate 2020010118 --comrot /some_large_disk_area/Joe.Schmo/comrot --expdir /some_safe_disk_area/Joe.Schmo/expdir --resdetatmos 384 --resensatmos 192 --nens 80 --gfs_cyc 4 Example ``setup_expt.py`` on Orion: :: - Orion-login-3$ ./setup_expt.py gfs cycled --pslot test --idate 2022010118 --edate 2022010200 --resdet 192 --resens 96 --nens 80 --comrot /work/noaa/stmp/jschmo/comrot --expdir /work/noaa/global/jschmo/expdir + Orion-login-3$ ./setup_expt.py gfs cycled --pslot test --idate 2022010118 --edate 2022010200 --resdetatmos 192 --resensatmos 96 --nens 80 --comrot /work/noaa/stmp/jschmo/comrot --expdir /work/noaa/global/jschmo/expdir EDITED: /work/noaa/global/jschmo/expdir/test/config.base as per user input. EDITED: /work/noaa/global/jschmo/expdir/test/config.aeroanl as per user input. EDITED: /work/noaa/global/jschmo/expdir/test/config.ocnanal as per user input. @@ -195,7 +197,7 @@ What happens if I run ``setup_expt.py`` again for an experiment that already exi :: - Orion-login-3$ ./setup_expt.py gfs cycled --pslot test --idate 2022010118 --edate 2022010200 --resdet 192 --resens 96 --nens 80 --comrot /work/noaa/stmp/jschmo/comrot --expdir /work/noaa/global/jschmo/expdir + Orion-login-3$ ./setup_expt.py gfs cycled --pslot test --idate 2022010118 --edate 2022010200 --resdetatmos 192 --resensatmos 96 --nens 80 --comrot /work/noaa/stmp/jschmo/comrot --expdir /work/noaa/global/jschmo/expdir directory already exists in /work/noaa/stmp/jschmo/comrot/test From 31502f40138eadd74412eac14e3b28298d33b5dc Mon Sep 17 00:00:00 2001 From: Rahul Mahajan Date: Tue, 9 Jan 2024 12:44:04 -0500 Subject: [PATCH 12/18] update default resdetocean --- workflow/setup_expt.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/workflow/setup_expt.py b/workflow/setup_expt.py index fe94c9ca3d7..b42c6630e1f 100755 --- a/workflow/setup_expt.py +++ b/workflow/setup_expt.py @@ -391,7 +391,7 @@ def _common_args(parser): parser.add_argument('--resdetatmos', help='atmosphere resolution of the deterministic model forecast', type=int, required=False, default=384) parser.add_argument('--resdetocean', help='ocean resolution of the deterministic model forecast', - type=float, required=False, default=-1.0) # -1.0 means determine from resdetatmos (limited combinations will be available) + type=float, required=False, default=0.0) # 0.0 (or lower) means determine from resdetatmos (limited combinations will be available) parser.add_argument('--comrot', help='full path to COMROT', type=str, required=False, default=os.getenv('HOME')) parser.add_argument('--expdir', help='full path to EXPDIR', @@ -547,7 +547,7 @@ def main(*argv): validate_user_request(host, user_inputs) # Determine ocean resolution if not provided - if user_inputs.resdetocean < 0: + if user_inputs.resdetocean <= 0: user_inputs.resdetocean = get_ocean_resolution(user_inputs.resdetatmos) comrot = os.path.join(user_inputs.comrot, user_inputs.pslot) From f6bb1d86214117f434a2c06cddac9014100f63af Mon Sep 17 00:00:00 2001 From: "Kate.Friedman" Date: Wed, 10 Jan 2024 19:01:42 +0000 Subject: [PATCH 13/18] Updated some scripts for COM directories Updates in some JJOB and ex-scripts remove use of ROTDIR and replace with appropriate COM_* directories. Refs #2061 --- jobs/JGDAS_GLOBAL_OCEAN_ANALYSIS_BMAT | 6 ++++- jobs/JGDAS_GLOBAL_OCEAN_ANALYSIS_BMAT_VRFY | 6 ++++- jobs/JGFS_ATMOS_FSU_GENESIS | 31 +++++++++------------- jobs/JGLOBAL_ATMOS_POST_MANAGER | 12 ++------- scripts/exglobal_atmos_pmgr.sh | 2 +- 5 files changed, 25 insertions(+), 32 deletions(-) diff --git a/jobs/JGDAS_GLOBAL_OCEAN_ANALYSIS_BMAT b/jobs/JGDAS_GLOBAL_OCEAN_ANALYSIS_BMAT index a1ecc116eab..4a8242abfb8 100755 --- a/jobs/JGDAS_GLOBAL_OCEAN_ANALYSIS_BMAT +++ b/jobs/JGDAS_GLOBAL_OCEAN_ANALYSIS_BMAT @@ -15,7 +15,11 @@ source "${HOMEgfs}/ush/jjob_header.sh" -e "ocnanalrun" -c "base ocnanal ocnanalr # Begin JOB SPECIFIC work ############################################## -export COMOUT=${COMOUT:-${ROTDIR}/${CDUMP}.${PDY}/${cyc}/ocean} +YMD=${PDY} HH=${cyc} generate_com -rx COM_OCEAN_ANALYSIS + +mkdir -p "${COM_OCEAN_ANALYSIS}" + +export COMOUT=${COM_OCEAN_ANALYSIS} ############################################################### # Run relevant script diff --git a/jobs/JGDAS_GLOBAL_OCEAN_ANALYSIS_BMAT_VRFY b/jobs/JGDAS_GLOBAL_OCEAN_ANALYSIS_BMAT_VRFY index 08e7da60c08..4f4251b34f1 100755 --- a/jobs/JGDAS_GLOBAL_OCEAN_ANALYSIS_BMAT_VRFY +++ b/jobs/JGDAS_GLOBAL_OCEAN_ANALYSIS_BMAT_VRFY @@ -14,7 +14,11 @@ source "${HOMEgfs}/ush/jjob_header.sh" -e "ocnanalrun" -c "base ocnanal ocnanalr # Begin JOB SPECIFIC work ############################################## -export COMOUT=${COMOUT:-${ROTDIR}/${CDUMP}.${PDY}/${cyc}/ocean} +YMD=${PDY} HH=${cyc} generate_com -rx COM_OCEAN_ANALYSIS + +mkdir -p "${COM_OCEAN_ANALYSIS}" + +export COMOUT=${COM_OCEAN_ANALYSIS} ############################################################### # Run relevant script diff --git a/jobs/JGFS_ATMOS_FSU_GENESIS b/jobs/JGFS_ATMOS_FSU_GENESIS index cc730e21bba..8b1600de881 100755 --- a/jobs/JGFS_ATMOS_FSU_GENESIS +++ b/jobs/JGFS_ATMOS_FSU_GENESIS @@ -33,25 +33,18 @@ export PYTHONPATH=${USHens_tracker}/FSUgenesisPY:${PYTHONPATH} ############################################## # Define COM and Data directories ############################################## -export COMIN=${ROTDIR}/${RUN}.${PDY}/${cyc}/${COMPONENT} -export COMINgfs=${COMIN} -#export gfsdir=${COMINgfs} -export gfsdir=${ROTDIR} -export COMINgdas=${COMIN} -export gdasdir=${COMINgdas} -export COMOUT=${ROTDIR}/${RUN}.${PDY}/${cyc}/${COMPONENT} -export COMINsyn=${COMINsyn:-$(compath.py ${envir}/com/gfs/${gfs_ver})/syndat} - -if [ ${RUN_ENVIR} = "nco" ]; then - export COMOUThur=${COMROOTp1}/hur/${envir}/global - export COMOUTatcf=${COMROOTp1}/nhc/${envir}/atcf - mkdir -m 775 -p ${COMOUThur} ${COMOUTatcf} -else -# export COMOUThur=$COMOUT -# export COMOUTatcf=$COMOUT - export COMOUThur=${DATA} - export COMOUTatcf=${DATA} -fi +YMD=${PDY} HH=${cyc} generate_com -rx COM_ATMOS_GENESIS +YMD=${PDY} HH=${cyc} GRID="0p25" generate_com -rx COM_ATMOS_GRIB_0p25:COM_ATMOS_GRIB_GRID_TMPL + +# The following variables are used by the tracker scripts which are outside +# of global-workflow and therefore can't be standardized at this time +export COMIN=${COM_ATMOS_GRIB_0p25} +export gfsdir=${COM_ATMOS_GRIB_0p25} +export COMINgfs=${COM_ATMOS_GRIB_0p25} + +export COMOUT=${COM_ATMOS_GENESIS} + +export COMINsyn=${COMINsyn:-$(compath.py "${envir}/com/gfs/${gfs_ver}")/syndat} ############################################## # Run relevant script diff --git a/jobs/JGLOBAL_ATMOS_POST_MANAGER b/jobs/JGLOBAL_ATMOS_POST_MANAGER index 1d82537dcab..3bbc258ce05 100755 --- a/jobs/JGLOBAL_ATMOS_POST_MANAGER +++ b/jobs/JGLOBAL_ATMOS_POST_MANAGER @@ -12,12 +12,6 @@ source "${HOMEgfs}/ush/jjob_header.sh" -e "post" -c "base post" export NET=${NET:-gfs} export RUN=${RUN:-gfs} - -#################################### -# Specify version numbers -#################################### -export gfs_ver=${gfs_ver:-v15.0.0} - #################################### # Specify Execution Areas #################################### @@ -32,10 +26,8 @@ export USHgfs=${HOMEgfs:-${HOMEgfs}/ush} ########################### export EXT_FCST=NO -export ROTDIR=${ROTDIR:-${COMROOT:?}/${NET}/${envir}} -export COMIN=${COMIN:-${ROTDIR}/${RUN}.${PDY}/${cyc}/atmos} -export COMOUT=${COMOUT:-${ROTDIR}/${RUN}.${PDY}/${cyc}/atmos} - +YMD=${PDY} HH=${cyc} generate_com -rx COM_ATMOS_HISTORY +YMD=${PDY} HH=${cyc} generate_com -rx COM_ATMOS_GRIB ######################################################## # Execute the script. diff --git a/scripts/exglobal_atmos_pmgr.sh b/scripts/exglobal_atmos_pmgr.sh index 6e4c2ed3f49..86afed962e0 100755 --- a/scripts/exglobal_atmos_pmgr.sh +++ b/scripts/exglobal_atmos_pmgr.sh @@ -43,7 +43,7 @@ icnt=1 while [ $icnt -lt 1000 ]; do for fhr in $postjobs; do fhr3=$(printf "%03d" $fhr) - if [ -s ${COMIN}/${RUN}.${cycle}.logf${fhr}.txt -o -s ${COMIN}/${RUN}.${cycle}.logf${fhr3}.txt ]; then + if [ -s ${COM_ATMOS_HISTORY}/${RUN}.${cycle}.atm.logf${fhr3}.txt ]; then if [ $fhr -eq 0 ]; then #### ecflow_client --event release_${RUN}_postanl ecflow_client --event release_postanl From ab031216f86492dbbd07a4232ccca87add326885 Mon Sep 17 00:00:00 2001 From: Kate Friedman Date: Wed, 10 Jan 2024 15:35:20 -0500 Subject: [PATCH 14/18] Update jobs/JGLOBAL_ATMOS_POST_MANAGER Remove unneeded `generate_com` command. Refs #2061 Co-authored-by: Walter Kolczynski - NOAA --- jobs/JGLOBAL_ATMOS_POST_MANAGER | 1 - 1 file changed, 1 deletion(-) diff --git a/jobs/JGLOBAL_ATMOS_POST_MANAGER b/jobs/JGLOBAL_ATMOS_POST_MANAGER index 3bbc258ce05..7c726bc2adb 100755 --- a/jobs/JGLOBAL_ATMOS_POST_MANAGER +++ b/jobs/JGLOBAL_ATMOS_POST_MANAGER @@ -27,7 +27,6 @@ export USHgfs=${HOMEgfs:-${HOMEgfs}/ush} export EXT_FCST=NO YMD=${PDY} HH=${cyc} generate_com -rx COM_ATMOS_HISTORY -YMD=${PDY} HH=${cyc} generate_com -rx COM_ATMOS_GRIB ######################################################## # Execute the script. From 15845c3daa791535d6cf237531b6b29c7f4f69e8 Mon Sep 17 00:00:00 2001 From: Rahul Mahajan Date: Wed, 10 Jan 2024 17:00:00 -0500 Subject: [PATCH 15/18] Update docs/doxygen/mainpage.h --- docs/doxygen/mainpage.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/doxygen/mainpage.h b/docs/doxygen/mainpage.h index 5d8ca875503..884dfddac9a 100644 --- a/docs/doxygen/mainpage.h +++ b/docs/doxygen/mainpage.h @@ -43,7 +43,7 @@ To setup an experiment, a python script \c setup_expt.py (located in \ --resdetatmos atmosphere resolution of the deterministic model forecast (default: 384) --resdetocean ocean resolution of the deterministic model forecast - (default: 0. [determined automatically based on atmosphere resolution]]]) + (default: 0. [determined automatically based on atmosphere resolution]) --resensatmos resolution of the ensemble model forecast (default: 192) --comrot full path to COMROT From 5b67aed61b4a2f17151ca594cf8203ba0b9296f1 Mon Sep 17 00:00:00 2001 From: "Kate.Friedman" Date: Thu, 11 Jan 2024 13:40:33 +0000 Subject: [PATCH 16/18] Change rotdir/ROTDIR to comroot/COMROOT in CI tests The comrot/COMROT variables should have been changed to comroot/COMROOT (with 2nd o/O) instead of rotdir/ROTDIR. Correcting now. Refs #2061 --- ci/cases/pr/C48_ATM.yaml | 2 +- ci/cases/pr/C48_S2SW.yaml | 2 +- ci/cases/pr/C48_S2SWA_gefs.yaml | 2 +- ci/cases/pr/C96C48_hybatmDA.yaml | 2 +- ci/cases/pr/C96_atm3DVar.yaml | 2 +- ci/cases/weekly/C384C192_hybatmda.yaml | 2 +- ci/cases/weekly/C384_S2SWA.yaml | 2 +- ci/cases/weekly/C384_atm3DVar.yaml | 2 +- ci/scripts/check_ci.sh | 2 +- 9 files changed, 9 insertions(+), 9 deletions(-) diff --git a/ci/cases/pr/C48_ATM.yaml b/ci/cases/pr/C48_ATM.yaml index ffd37409081..4d585647bbc 100644 --- a/ci/cases/pr/C48_ATM.yaml +++ b/ci/cases/pr/C48_ATM.yaml @@ -6,7 +6,7 @@ arguments: pslot: {{ 'pslot' | getenv }} app: ATM resdet: 48 - rotdir: {{ 'RUNTESTS' | getenv }}/ROTDIR + comroot: {{ 'RUNTESTS' | getenv }}/COMROOT expdir: {{ 'RUNTESTS' | getenv }}/EXPDIR idate: 2021032312 edate: 2021032312 diff --git a/ci/cases/pr/C48_S2SW.yaml b/ci/cases/pr/C48_S2SW.yaml index d7f781f1180..004b75a3082 100644 --- a/ci/cases/pr/C48_S2SW.yaml +++ b/ci/cases/pr/C48_S2SW.yaml @@ -6,7 +6,7 @@ arguments: pslot: {{ 'pslot' | getenv }} app: S2SW resdet: 48 - rotdir: {{ 'RUNTESTS' | getenv }}/ROTDIR + comroot: {{ 'RUNTESTS' | getenv }}/COMROOT expdir: {{ 'RUNTESTS' | getenv }}/EXPDIR idate: 2021032312 edate: 2021032312 diff --git a/ci/cases/pr/C48_S2SWA_gefs.yaml b/ci/cases/pr/C48_S2SWA_gefs.yaml index 0f6fc84fd4a..79aed919e52 100644 --- a/ci/cases/pr/C48_S2SWA_gefs.yaml +++ b/ci/cases/pr/C48_S2SWA_gefs.yaml @@ -10,7 +10,7 @@ arguments: nens: 2 gfs_cyc: 1 start: cold - rotdir: {{ 'RUNTESTS' | getenv }}/ROTDIR + comroot: {{ 'RUNTESTS' | getenv }}/COMROOT expdir: {{ 'RUNTESTS' | getenv }}/EXPDIR idate: 2021032312 edate: 2021032312 diff --git a/ci/cases/pr/C96C48_hybatmDA.yaml b/ci/cases/pr/C96C48_hybatmDA.yaml index 26096bf5052..9804449fe50 100644 --- a/ci/cases/pr/C96C48_hybatmDA.yaml +++ b/ci/cases/pr/C96C48_hybatmDA.yaml @@ -7,7 +7,7 @@ arguments: app: ATM resdet: 96 resens: 48 - rotdir: {{ 'RUNTESTS' | getenv }}/ROTDIR + comroot: {{ 'RUNTESTS' | getenv }}/COMROOT expdir: {{ 'RUNTESTS' | getenv }}/EXPDIR icsdir: {{ 'ICSDIR_ROOT' | getenv }}/C96C48 idate: 2021122018 diff --git a/ci/cases/pr/C96_atm3DVar.yaml b/ci/cases/pr/C96_atm3DVar.yaml index ba746a1214a..dbd4ec38ea3 100644 --- a/ci/cases/pr/C96_atm3DVar.yaml +++ b/ci/cases/pr/C96_atm3DVar.yaml @@ -6,7 +6,7 @@ arguments: pslot: {{ 'pslot' | getenv }} app: ATM resdet: 96 - rotdir: {{ 'RUNTESTS' | getenv }}/ROTDIR + comroot: {{ 'RUNTESTS' | getenv }}/COMROOT expdir: {{ 'RUNTESTS' | getenv }}/EXPDIR icsdir: ${ICSDIR_ROOT}/C96C48 idate: 2021122018 diff --git a/ci/cases/weekly/C384C192_hybatmda.yaml b/ci/cases/weekly/C384C192_hybatmda.yaml index c7a9db6fd05..b6fff26344c 100644 --- a/ci/cases/weekly/C384C192_hybatmda.yaml +++ b/ci/cases/weekly/C384C192_hybatmda.yaml @@ -7,7 +7,7 @@ arguments: app: ATM resdet: 384 resens: 192 - rotdir: {{ 'RUNTESTS' | getenv }}/ROTDIR + comroot: {{ 'RUNTESTS' | getenv }}/COMROOT expdir: {{ 'RUNTESTS' | getenv }}/EXPDIR icsdir: {{ 'ICSDIR_ROOT' | getenv }}/C384C192 idate: 2023040118 diff --git a/ci/cases/weekly/C384_S2SWA.yaml b/ci/cases/weekly/C384_S2SWA.yaml index 573295a4522..5f1f5c3dc8b 100644 --- a/ci/cases/weekly/C384_S2SWA.yaml +++ b/ci/cases/weekly/C384_S2SWA.yaml @@ -6,7 +6,7 @@ arguments: pslot: {{ 'pslot' | getenv }} app: S2SWA resdet: 384 - rotdir: {{ 'RUNTESTS' | getenv }}/ROTDIR + comroot: {{ 'RUNTESTS' | getenv }}/COMROOT expdir: {{ 'RUNTESTS' | getenv }}/EXPDIR idate: 2016070100 edate: 2016070100 diff --git a/ci/cases/weekly/C384_atm3DVar.yaml b/ci/cases/weekly/C384_atm3DVar.yaml index 1bec5041938..98a7a333c15 100644 --- a/ci/cases/weekly/C384_atm3DVar.yaml +++ b/ci/cases/weekly/C384_atm3DVar.yaml @@ -7,7 +7,7 @@ arguments: app: ATM resdet: 384 resens: 192 - rotdir: {{ 'RUNTESTS' | getenv }}/ROTDIR + comroot: {{ 'RUNTESTS' | getenv }}/COMROOT expdir: {{ 'RUNTESTS' | getenv }}/EXPDIR icsdir: {{ 'ICSDIR_ROOT' | getenv }}/C384C192 idate: 2023040118 diff --git a/ci/scripts/check_ci.sh b/ci/scripts/check_ci.sh index 496021cd621..cda2d4e9f2f 100755 --- a/ci/scripts/check_ci.sh +++ b/ci/scripts/check_ci.sh @@ -149,7 +149,7 @@ for pr in ${pr_list}; do if [[ "${num_done}" -eq "${num_cycles}" ]]; then #Remove Experment cases that completed successfully rm -Rf "${pslot_dir}" - rm -Rf "${pr_dir}/RUNTESTS/ROTDIR/${pslot}" + rm -Rf "${pr_dir}/RUNTESTS/COMROOT/${pslot}" rm -f "${output_ci_single}" # echo "\`\`\`" > "${output_ci_single}" DATE=$(date +'%D %r') From 236f1a546bc7b9820ce00d53a6f41e4f766c7efd Mon Sep 17 00:00:00 2001 From: "Kate.Friedman" Date: Thu, 11 Jan 2024 13:43:12 +0000 Subject: [PATCH 17/18] Change ROTDIR to COMROOT in CI check script Refs #2061 --- ci/scripts/run-check_ci.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/ci/scripts/run-check_ci.sh b/ci/scripts/run-check_ci.sh index 00130d522ef..5a909c1c64e 100755 --- a/ci/scripts/run-check_ci.sh +++ b/ci/scripts/run-check_ci.sh @@ -12,12 +12,12 @@ pslot=${2:-${pslot:-?}} # Name of the experiment being tested by this scr # TEST_DIR contains 2 directories; # 1. HOMEgfs: clone of the global-workflow -# 2. RUNTESTS: A directory containing EXPDIR and ROTDIR for experiments +# 2. RUNTESTS: A directory containing EXPDIR and COMROOT for experiments # # e.g. $> tree ./TEST_DIR # ./TEST_DIR # ├── HOMEgfs # └── RUNTESTS -# ├── ROTDIR +# ├── COMROOT # │   └── ${pslot} # └── EXPDIR # └── ${pslot} From 19131b6d9bd308c90e4adf559c641a0ac0ef728b Mon Sep 17 00:00:00 2001 From: Rahul Mahajan Date: Thu, 11 Jan 2024 11:17:27 -0500 Subject: [PATCH 18/18] Update ci/cases/pr/C96_atm3DVar.yaml --- ci/cases/pr/C96_atm3DVar.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ci/cases/pr/C96_atm3DVar.yaml b/ci/cases/pr/C96_atm3DVar.yaml index b068ebda27d..c1006479afe 100644 --- a/ci/cases/pr/C96_atm3DVar.yaml +++ b/ci/cases/pr/C96_atm3DVar.yaml @@ -8,7 +8,7 @@ arguments: resdetatmos: 96 comrot: {{ 'RUNTESTS' | getenv }}/COMROT expdir: {{ 'RUNTESTS' | getenv }}/EXPDIR - icsdir: {{ 'ICSDIR_ROOT' | getenv ]}/C96C48 + icsdir: {{ 'ICSDIR_ROOT' | getenv }}/C96C48 idate: 2021122018 edate: 2021122106 nens: 0