From 0d67a22e1c8183e745f97fc5dd23ea0661cff698 Mon Sep 17 00:00:00 2001 From: Joshua Kublnick Date: Wed, 11 Feb 2026 11:24:14 -0500 Subject: [PATCH 01/32] AquaPlanet Test case introduction, cloning, and building instructions --- doc/UsersGuide/source/AquaPlanet.rst | 109 +++++++++++++++++++++++++++ 1 file changed, 109 insertions(+) create mode 100644 doc/UsersGuide/source/AquaPlanet.rst diff --git a/doc/UsersGuide/source/AquaPlanet.rst b/doc/UsersGuide/source/AquaPlanet.rst new file mode 100644 index 0000000000..6f52c2518b --- /dev/null +++ b/doc/UsersGuide/source/AquaPlanet.rst @@ -0,0 +1,109 @@ +.. role:: raw-html(raw) + :format: html + +.. _aquaplanet: + +********************* +AquaPlanet Test Case +********************* + +The AquaPlanet test case is an idealized atmosphere-only forecast configuration designed to study atmospheric dynamics in a simplified Earth-like setting where all land is replaced with ocean. This configuration removes the complexity of land-surface interactions, topography, and regional variations, allowing researchers to focus on fundamental atmospheric processes such as tropical convection, Hadley circulation, jet stream dynamics, and the global energy budget. + +The test case runs at C48 resolution with the ``FV3_GFS_v17_p8_ugwpv1`` physics suite. Initial conditions are created by modifying standard GFS data to represent an aquaplanet configuration: sea surface temperatures (SST) follow an idealized latitudinal profile, all land is converted to ocean, topography is set to zero, and sea ice is removed. The atmospheric initial state is configured with idealized vertical profiles of temperature, humidity, and winds appropriate for an aquaplanet simulation. + +A key feature of this configuration is the 90-day spin-up period required to allow the model to adjust from its initial state to a balanced aquaplanet climate. After spin-up, the simulation can be run for extended periods to study seasonal variations, climate statistics, and atmospheric phenomena in the absence of land-surface influences. + +============================ +Obtaining Data for HSD Cases +============================ + +.. include:: ./doc-snippets/hsd_data.rst + +.. _setup-aquaplanet: + +========================================== +Setting Up the AquaPlanet Experiment +========================================== + +This section describes how to set up the AquaPlanet experiment from scratch. The process involves modifying input data files, running a 90-day spin-up simulation, and then running experiments. + +Clone Required Repositories +---------------------------- + +First, clone the UFS Weather Model and UFS_UTILS repositories: + +.. code-block:: console + + git clone https://github.com/ufs-community/ufs-weather-model.git + cd ufs-weather-model/ + git submodule update --init --recursive + +Then clone the UFS_UTILS repository: + +.. code-block:: console + + git clone https://github.com/ufs-community/UFS_UTILS.git + cd UFS_UTILS/ + git submodule update --init --recursive + +Finally, clone the aquaplanet tools repository: + +.. code-block:: console + + git clone https://github.com/RatkoVasic-NOAA/Aquaplanet + +Build UFS Weather Model +------------------------ + +Navigate to the UFS Weather Model test directory and configure the regression test system to compile the model: + +.. code-block:: console + + cd ufs-weather-model/tests/ + +Edit ``rt.conf`` to include only the following two lines: + +.. code-block:: console + + COMPILE | atm_dyn32 | intel | -DAPP=ATM -DCCPP_SUITES=FV3_GFS_v17_p8_ugwpv1 -D32BIT=ON | | fv3 | + RUN | control_c48 | | baseline | + +Execute the regression test to compile the model and create a baseline run directory: + +.. code-block:: console + + ./rt.sh -a -k + +Replace ```` with your project code (e.g., ``epic``). Save the location of the ``control_c48_intel`` run directory for later use. + +Build UFS_UTILS +--------------- + +Compile the UFS_UTILS tools: + +.. code-block:: console + + cd UFS_UTILS/ + module purge + module use $PWD/modulefiles + module load build..intelllvm + ./build_all.sh + +Replace ```` with your system name (e.g., ``ursa``). + +Link the fix directories: + +.. code-block:: console + + cd fix/ + ./link_fixdirs.sh emc + +Since orography files need to be edited, remove the link and copy the files: + +.. code-block:: console + + rm orog + mkdir orog + cp /scratch3/NCEPDEV/global/role.glopara/fix/orog/20240917/*.nc orog/. + cp /scratch3/NCEPDEV/global/role.glopara/fix/orog/20240917/*.dat orog/. + cp -r /scratch3/NCEPDEV/global/role.glopara/fix/orog/20240917/C48/ orog/. \ No newline at end of file From c20618e49c456978e997e602280aa9bbd47d93bc Mon Sep 17 00:00:00 2001 From: Joshua Kublnick Date: Fri, 13 Feb 2026 12:17:16 -0500 Subject: [PATCH 02/32] Summary, obtaining data, running, cloning, config, running tests and example --- doc/UsersGuide/source/AquaPlanet.rst | 97 +++++-------------- .../source/doc-snippets/hsd_data.rst | 2 +- .../source/doc-snippets/hsd_run_multiple.rst | 2 +- .../source/doc-snippets/hsd_run_tests.rst | 2 +- 4 files changed, 27 insertions(+), 76 deletions(-) diff --git a/doc/UsersGuide/source/AquaPlanet.rst b/doc/UsersGuide/source/AquaPlanet.rst index 6f52c2518b..c2db2982e0 100644 --- a/doc/UsersGuide/source/AquaPlanet.rst +++ b/doc/UsersGuide/source/AquaPlanet.rst @@ -19,91 +19,42 @@ Obtaining Data for HSD Cases .. include:: ./doc-snippets/hsd_data.rst -.. _setup-aquaplanet: +.. _run-aquaplanet: -========================================== -Setting Up the AquaPlanet Experiment -========================================== +============================== +Running the AquaPlanet Case +============================== -This section describes how to set up the AquaPlanet experiment from scratch. The process involves modifying input data files, running a 90-day spin-up simulation, and then running experiments. +This section explains how to run the AquaPlanet case using the ``ufs_test.sh`` script with pre-staged initial conditions. This is the recommended way to run the case for most users. -Clone Required Repositories ----------------------------- +Clone the Repository +-------------------- -First, clone the UFS Weather Model and UFS_UTILS repositories: +.. include:: ./doc-snippets/clone_hsd.rst -.. code-block:: console - - git clone https://github.com/ufs-community/ufs-weather-model.git - cd ufs-weather-model/ - git submodule update --init --recursive - -Then clone the UFS_UTILS repository: - -.. code-block:: console - - git clone https://github.com/ufs-community/UFS_UTILS.git - cd UFS_UTILS/ - git submodule update --init --recursive - -Finally, clone the aquaplanet tools repository: - -.. code-block:: console - - git clone https://github.com/RatkoVasic-NOAA/Aquaplanet - -Build UFS Weather Model ------------------------- - -Navigate to the UFS Weather Model test directory and configure the regression test system to compile the model: - -.. code-block:: console +Machine Configuration +--------------------- - cd ufs-weather-model/tests/ +.. include:: ./doc-snippets/hsd_machine_config.rst -Edit ``rt.conf`` to include only the following two lines: +Running Tests +------------- -.. code-block:: console - - COMPILE | atm_dyn32 | intel | -DAPP=ATM -DCCPP_SUITES=FV3_GFS_v17_p8_ugwpv1 -D32BIT=ON | | fv3 | - RUN | control_c48 | | baseline | - -Execute the regression test to compile the model and create a baseline run directory: - -.. code-block:: console - - ./rt.sh -a -k - -Replace ```` with your project code (e.g., ``epic``). Save the location of the ``control_c48_intel`` run directory for later use. +.. include:: ./doc-snippets/hsd_run_tests.rst -Build UFS_UTILS ---------------- +Example +^^^^^^^ -Compile the UFS_UTILS tools: +Users with access to the ``epic`` account can run the ``aquaplanet`` test case with the ``intel`` compiler on :term:`RDHPCS` where they have access using the following command: .. code-block:: console - cd UFS_UTILS/ - module purge - module use $PWD/modulefiles - module load build..intelllvm - ./build_all.sh + ./ufs_test.sh -a epic -s -c -k -r -n "aquaplanet intel" -Replace ```` with your system name (e.g., ``ursa``). - -Link the fix directories: - -.. code-block:: console - - cd fix/ - ./link_fixdirs.sh emc - -Since orography files need to be edited, remove the link and copy the files: - -.. code-block:: console +Where: - rm orog - mkdir orog - cp /scratch3/NCEPDEV/global/role.glopara/fix/orog/20240917/*.nc orog/. - cp /scratch3/NCEPDEV/global/role.glopara/fix/orog/20240917/*.dat orog/. - cp -r /scratch3/NCEPDEV/global/role.glopara/fix/orog/20240917/C48/ orog/. \ No newline at end of file +- ``-s``: use tests-dev, symlink sharable test scripts +- ``-c``: prevents scripts from comparing with previous results +- ``-k``: keep run directory +- ``-r``: use rocoto scheduler (``-e`` will use ecFlow) +- ``-n "aquaplanet intel"``: use the aquaplanet test case with intel compiler \ No newline at end of file diff --git a/doc/UsersGuide/source/doc-snippets/hsd_data.rst b/doc/UsersGuide/source/doc-snippets/hsd_data.rst index c56273d2a7..e53530fe86 100644 --- a/doc/UsersGuide/source/doc-snippets/hsd_data.rst +++ b/doc/UsersGuide/source/doc-snippets/hsd_data.rst @@ -2,5 +2,5 @@ Data for the HSD cases is already staged on Tier-1 platforms at the ``INPUTDATA_ .. code-block:: console - wget https://noaa-ufs-htf-pds.s3.amazonaws.com/develop-20250530/HSD_fix_files_and_case_data.tar.gz + wget https://noaa-ufs-htf-pds.s3.amazonaws.com/index.html#develop-20260212/ tar xvfz HSD_fix_files_and_case_data.tar.gz diff --git a/doc/UsersGuide/source/doc-snippets/hsd_run_multiple.rst b/doc/UsersGuide/source/doc-snippets/hsd_run_multiple.rst index 81dd048565..24f81b7fa9 100644 --- a/doc/UsersGuide/source/doc-snippets/hsd_run_multiple.rst +++ b/doc/UsersGuide/source/doc-snippets/hsd_run_multiple.rst @@ -4,7 +4,7 @@ To run multiple cases at once, modify ``ufs_test.yaml`` to contain only a subset ./ufs_test.sh -a epic -s -c -k -r -l ufs_test.yaml -Alternatively, users may copy the sections for ``2020_CAPE``/ ``baroclinic_wave``/ ``tropical_cyclone`` tests into a new YAML file (e.g., ``my_test_cases.yaml``) to call via ``ufs_test.sh``: +Alternatively, users may copy the sections for ``2020_CAPE``/ ``baroclinic_wave``/ ``tropical_cyclone``/ ``aquaplanet``/ tests into a new YAML file (e.g., ``my_test_cases.yaml``) to call via ``ufs_test.sh``: .. code-block:: console diff --git a/doc/UsersGuide/source/doc-snippets/hsd_run_tests.rst b/doc/UsersGuide/source/doc-snippets/hsd_run_tests.rst index fe047a3dfb..93d5286894 100644 --- a/doc/UsersGuide/source/doc-snippets/hsd_run_tests.rst +++ b/doc/UsersGuide/source/doc-snippets/hsd_run_tests.rst @@ -8,7 +8,7 @@ Launch tests from the ``${UFS_WM}/tests-dev`` directory with the following comma where: * ````: Account/project number for batch jobs. -* ````: Name of the test case (e.g., ``2020_CAPE``, ``baroclinic_wave``, or ``tropical_cyclone``). +* ````: Name of the test case (e.g., ``2020_CAPE``, ``baroclinic_wave``, ``aquaplanet``, or ``tropical_cyclone``). * ````: Compiler used for the tests (``intel`` or ``gnu``). **Command-line Options:** From e808e02df2da0fddef7491006f02bf2b151339c3 Mon Sep 17 00:00:00 2001 From: Joshua Kublnick Date: Fri, 13 Feb 2026 13:01:53 -0500 Subject: [PATCH 03/32] Adding sections for running extended experiments and checking results --- doc/UsersGuide/source/AquaPlanet.rst | 55 +++++++++++++++++++++++++++- 1 file changed, 54 insertions(+), 1 deletion(-) diff --git a/doc/UsersGuide/source/AquaPlanet.rst b/doc/UsersGuide/source/AquaPlanet.rst index c2db2982e0..250cc9a736 100644 --- a/doc/UsersGuide/source/AquaPlanet.rst +++ b/doc/UsersGuide/source/AquaPlanet.rst @@ -57,4 +57,57 @@ Where: - ``-c``: prevents scripts from comparing with previous results - ``-k``: keep run directory - ``-r``: use rocoto scheduler (``-e`` will use ecFlow) -- ``-n "aquaplanet intel"``: use the aquaplanet test case with intel compiler \ No newline at end of file +- ``-n "aquaplanet intel"``: use the aquaplanet test case with intel compiler + +Running Extended Experiments +----------------------------- + +After running the base test case, users can extend the simulation by running multiple restart segments. For example, to run a 1-year experiment broken into four 3-month segments: + +Navigate to the run directory (linked as ``./tests-dev/run_dir``). + +Use the ``input.nml`` file configured for restart runs (with ``warm_start = .true.``). + +For each 3-month segment, update ``fhrot`` and ``nhours_fcst`` in ``model_configure``: + +**First segment (months 4-6):** + +.. code-block:: console + + fhrot: 2160 + nhours_fcst: 4320 + +**Second segment (months 7-9):** + +.. code-block:: console + + fhrot: 4320 + nhours_fcst: 6480 + +**Third segment (months 10-12):** + +.. code-block:: console + + fhrot: 6480 + nhours_fcst: 8640 + +After each segment completes, rename and move restart files to the ``INPUT`` directory: + +.. code-block:: console + + cd RESTART/ + for file in 20*.060000.*.nc; do mv "$file" "${file#20*.060000.}"; done + mv 20*.060000.coupler.res coupler.res + mv * ../INPUT/. + +Checking Results +---------------- + +.. include:: ./doc-snippets/hsd_check_results.rst + +For example, to monitor progress or check results for the ``aquaplanet`` case, run: + +.. code-block:: console + + tail -f ${UFS_WM}/tests-dev/run_dir/aquaplanet_intel/err + tail -f ${UFS_WM}/tests-dev/run_dir/aquaplanet_intel/out \ No newline at end of file From 89839c643bcb6a6660592ad2a7fcce34302f8d11 Mon Sep 17 00:00:00 2001 From: Joshua Kublnick Date: Fri, 13 Feb 2026 13:16:34 -0500 Subject: [PATCH 04/32] Plotting section --- doc/UsersGuide/source/AquaPlanet.rst | 45 +++++++++++++++++++++++++++- 1 file changed, 44 insertions(+), 1 deletion(-) diff --git a/doc/UsersGuide/source/AquaPlanet.rst b/doc/UsersGuide/source/AquaPlanet.rst index 250cc9a736..be33c55233 100644 --- a/doc/UsersGuide/source/AquaPlanet.rst +++ b/doc/UsersGuide/source/AquaPlanet.rst @@ -110,4 +110,47 @@ For example, to monitor progress or check results for the ``aquaplanet`` case, r .. code-block:: console tail -f ${UFS_WM}/tests-dev/run_dir/aquaplanet_intel/err - tail -f ${UFS_WM}/tests-dev/run_dir/aquaplanet_intel/out \ No newline at end of file + tail -f ${UFS_WM}/tests-dev/run_dir/aquaplanet_intel/out + + .. _plotting-aquaplanet: + +======================= +Plotting Script +======================= + +A plotting script is available to generate seasonal mean plots for key atmospheric variables. The script is located at: + +.. code-block:: console + + ./tests-dev/test_cases/utils/plot_aq.sh + +.. note:: + + This plotting script is currently configured to run on Hera and Ursa only. + +By default, this script creates seasonal means for three variables: + +- Jet stream characteristics +- Precipitation patterns +- Temperature at 500 mb + +The script uses staged data from a 1-year control simulation. If you want to use this dataset on other machines (which have access to :term:`HPSS`), you can retrieve it from HPSS: + +.. code-block:: console + + /5year/NCEPDEV/emc-meso/Ratko.Vasic/AQUAPLANET/1yr-results.tar + +Customizing the Plotting Script +-------------------------------- + +To use the plotting script with user-generated data: + +1. Copy the script to your run directory. +2. Edit the variable ``out_pth`` to point to your output location: + + .. code-block:: bash + + out_pth=./ + +3. Adjust the seasonal timing variables (``winter_start``, ``spring_start``, etc.), which are given in hours from the start of the 90-day spin-up run (hour 0). +4. Modify ``season_len`` to set the length of each season in days (typically 90 days, but can be set to 365 for annual means). \ No newline at end of file From ee1dc335f4a1bd7e7afe15b88b0c008e632097a7 Mon Sep 17 00:00:00 2001 From: Joshua Kublnick Date: Fri, 13 Feb 2026 13:27:15 -0500 Subject: [PATCH 05/32] Setting up case from scratch cloning section --- doc/UsersGuide/source/AquaPlanet.rst | 48 +++++++++++++++++++++++++++- 1 file changed, 47 insertions(+), 1 deletion(-) diff --git a/doc/UsersGuide/source/AquaPlanet.rst b/doc/UsersGuide/source/AquaPlanet.rst index be33c55233..1cc626847e 100644 --- a/doc/UsersGuide/source/AquaPlanet.rst +++ b/doc/UsersGuide/source/AquaPlanet.rst @@ -153,4 +153,50 @@ To use the plotting script with user-generated data: out_pth=./ 3. Adjust the seasonal timing variables (``winter_start``, ``spring_start``, etc.), which are given in hours from the start of the 90-day spin-up run (hour 0). -4. Modify ``season_len`` to set the length of each season in days (typically 90 days, but can be set to 365 for annual means). \ No newline at end of file +4. Modify ``season_len`` to set the length of each season in days (typically 90 days, but can be set to 365 for annual means). + +.. _setup-aquaplanet: + +========================================================== +Advanced: Setting Up the AquaPlanet Experiment from Scratch +========================================================== + +.. note:: + + This section is **optional**. Most users can run the AquaPlanet case using the ``ufs_test.sh`` method described above, which uses pre-staged initial conditions. The steps below are for advanced users who wish to create their own initial conditions from scratch. + +The from-scratch setup involves editing orography, SST, ice, and sea-land mask files to create an aquaplanet configuration, then running a 90-day spin-up to allow the model to reach a balanced state. This process produces the same initial conditions that are provided pre-staged for the standard test case. + +At a high level, the steps are: + +1. Build the UFS Weather Model and run a baseline ``control_c48`` test to generate a run directory. +2. Build UFS_UTILS and use ``gdas_init`` to generate raw atmospheric initial conditions. +3. Use the `Aquaplanet tools `_ to modify SST, ice, sea-land mask, orography, and atmospheric/surface profiles to represent an aquaplanet. +4. Regenerate initial conditions with the modified files. +5. Apply minor source code changes and recompile the model. +6. Run a 90-day spin-up simulation (in three 30-day restart segments). + +Clone Required Repositories +---------------------------- + +Clone the UFS Weather Model: + +.. code-block:: console + + git clone https://github.com/ufs-community/ufs-weather-model.git + cd ufs-weather-model/ + git submodule update --init --recursive + +Clone UFS_UTILS: + +.. code-block:: console + + git clone https://github.com/ufs-community/UFS_UTILS.git + cd UFS_UTILS/ + git submodule update --init --recursive + +Clone the aquaplanet tools: + +.. code-block:: console + + git clone https://github.com/RatkoVasic-NOAA/Aquaplanet From 2c3e56026c009e9f3838059e5fc778b03874577f Mon Sep 17 00:00:00 2001 From: Joshua Kublnick Date: Fri, 13 Feb 2026 14:06:25 -0500 Subject: [PATCH 06/32] Adding build sections for ufs_utils and UFSWM --- doc/UsersGuide/source/AquaPlanet.rst | 56 ++++++++++++++++++++++++++++ 1 file changed, 56 insertions(+) diff --git a/doc/UsersGuide/source/AquaPlanet.rst b/doc/UsersGuide/source/AquaPlanet.rst index 1cc626847e..82edffebba 100644 --- a/doc/UsersGuide/source/AquaPlanet.rst +++ b/doc/UsersGuide/source/AquaPlanet.rst @@ -200,3 +200,59 @@ Clone the aquaplanet tools: .. code-block:: console git clone https://github.com/RatkoVasic-NOAA/Aquaplanet + +Build UFS Weather Model +------------------------ + +Navigate to the UFS Weather Model test directory: + +.. code-block:: console + + cd ufs-weather-model/tests/ + +Edit ``rt.conf`` to include only the following two lines: + +.. code-block:: console + + COMPILE | atm_dyn32 | intel | -DAPP=ATM -DCCPP_SUITES=FV3_GFS_v17_p8_ugwpv1 -D32BIT=ON | | fv3 | + RUN | control_c48 | | baseline | + +Execute the regression test to compile the model and create a baseline run directory: + +.. code-block:: console + + ./rt.sh -a -k + +Replace ```` with your project code (e.g., ``epic``). Save the location of the ``control_c48_intel`` run directory for later use. + +Build UFS_UTILS +--------------- + +Compile the UFS_UTILS tools: + +.. code-block:: console + + cd UFS_UTILS/ + module purge + module use $PWD/modulefiles + module load build..intelllvm + ./build_all.sh + +Replace ```` with your system name (e.g., ``ursa``). + +Link the fix directories: + +.. code-block:: console + + cd fix/ + ./link_fixdirs.sh emc + +Since orography files need to be edited, remove the link and copy the files: + +.. code-block:: console + + rm orog + mkdir orog + cp /scratch3/NCEPDEV/global/role.glopara/fix/orog/20240917/*.nc orog/. + cp /scratch3/NCEPDEV/global/role.glopara/fix/orog/20240917/*.dat orog/. + cp -r /scratch3/NCEPDEV/global/role.glopara/fix/orog/20240917/C48/ orog/. \ No newline at end of file From 96b2766eb4fe8557e3cc08b4bac09eb33dbc9290 Mon Sep 17 00:00:00 2001 From: Joshua Kublnick Date: Fri, 13 Feb 2026 14:45:19 -0500 Subject: [PATCH 07/32] Generating intial conditions, creating SST profile, editing global ice data, and editing monthly ice data --- doc/UsersGuide/source/AquaPlanet.rst | 131 ++++++++++++++++++++++++++- 1 file changed, 130 insertions(+), 1 deletion(-) diff --git a/doc/UsersGuide/source/AquaPlanet.rst b/doc/UsersGuide/source/AquaPlanet.rst index 82edffebba..4e93853c47 100644 --- a/doc/UsersGuide/source/AquaPlanet.rst +++ b/doc/UsersGuide/source/AquaPlanet.rst @@ -255,4 +255,133 @@ Since orography files need to be edited, remove the link and copy the files: mkdir orog cp /scratch3/NCEPDEV/global/role.glopara/fix/orog/20240917/*.nc orog/. cp /scratch3/NCEPDEV/global/role.glopara/fix/orog/20240917/*.dat orog/. - cp -r /scratch3/NCEPDEV/global/role.glopara/fix/orog/20240917/C48/ orog/. \ No newline at end of file + cp -r /scratch3/NCEPDEV/global/role.glopara/fix/orog/20240917/C48/ orog/. + +Generate Initial Atmospheric Data +---------------------------------- + +Navigate to the ``gdas_init`` utility directory: + +.. code-block:: console + + cd ../util/gdas_init/ + +Edit the ``config`` file to set the following variables: + +.. code-block:: console + + EXTRACT_DIR=/input/ + EXTRACT_DATA=yes + yy=2025 + mm=10 + dd=15 + CRES_HIRES=C48 + OUTDIR=/output/ + +Edit the driver script (e.g., ``driver.ursa.sh``) to set: + +.. code-block:: console + + PROJECT_CODE= + +Comment out the line that removes the extract directory: + +.. code-block:: console + + # rm -fr $EXTRACT_DIR + +Run the driver script: + +.. code-block:: console + + ./driver..sh + +Check that results are properly generated in ``$EXTRACT_DIR`` and ``$OUTDIR``. + +Create Idealized SST Profile +----------------------------- + +Navigate to the SST profile tool directory: + +.. code-block:: console + + cd ~/Aquaplanet/sst-profile/ + +Copy the SST climatology file from your UFS Weather Model run directory: + +.. code-block:: console + + cp /RTGSST.1982.2012.monthly.clim.grb . + +Compile and run the tool: + +.. code-block:: console + + ./compile.sh + ./sst-profile.x + +Copy the modified file back to your run directory: + +.. code-block:: console + + cp new-RTGSST.1982.2012.monthly.clim.grb /RTGSST.1982.2012.monthly.clim.grb + +Edit Global Ice Data +-------------------- + +Navigate to the glacier tool directory: + +.. code-block:: console + + cd ~/Aquaplanet/glacier/ + +Copy the glacier file from your run directory: + +.. code-block:: console + + cp /global_glacier.2x2.grb . + +Compile and run the tool: + +.. code-block:: console + + ./compile.sh + ./glacier.x + +Copy the modified file back: + +.. code-block:: console + + cp new-global_glacier.2x2.grb /global_glacier.2x2.grb + +Edit Monthly Ice Data +--------------------- + +Navigate to the ice monthly tool directory: + +.. code-block:: console + + cd ~/Aquaplanet/ice-monthly/ + +Copy the ice climatology file from your run directory: + +.. code-block:: console + + cp /IMS-NIC.blended.ice.monthly.clim.grb . + +Compile and run the tool: + +.. code-block:: console + + ./compile.sh + ./ice-monthly.x + +.. note:: + + If the tool fails due to lack of memory, use the provided ``job_card`` file as a template and submit to a batch node. + +Copy the modified file back: + +.. code-block:: console + + cp new-IMS-NIC.blended.ice.monthly.clim.grb /IMS-NIC.blended.ice.monthly.clim.grb \ No newline at end of file From d46d7d603eb36d6829aabd1058887da38e133c24 Mon Sep 17 00:00:00 2001 From: Joshua Kublnick Date: Fri, 13 Feb 2026 15:10:06 -0500 Subject: [PATCH 08/32] Container updates --- doc/UsersGuide/source/HSDcontainer.rst | 40 +++++++++++++------------- 1 file changed, 20 insertions(+), 20 deletions(-) diff --git a/doc/UsersGuide/source/HSDcontainer.rst b/doc/UsersGuide/source/HSDcontainer.rst index f6df08f136..7518ab4c3b 100644 --- a/doc/UsersGuide/source/HSDcontainer.rst +++ b/doc/UsersGuide/source/HSDcontainer.rst @@ -11,10 +11,11 @@ This chapter provides instructions for running the Unified Forecast System (:ter The container includes: spack-stack, Intel’s runtime environment, ufs-weather-model repository, and the prebuilt executables for the HSD cases. These are all the components needed to run the HSD cases besides the HSD data, which is discussed later in this chapter. -This chapter provides instructions for building and running the Unified Forecast System UFS WM HSD cases using a container. Currently, users can select from the following cases: +This chapter provides instructions for building and running the Unified Forecast System UFS WM HSD cases using a container. Currently, users can select from the following cases: * The :ref:`July 2020 CAPE Case ` * The :ref:`Baroclinic Instability Case ` + * The :ref:`AquaPlanet Test Case ` * The :ref:`Idealized, Regional Tropical Cyclone Case ` .. attention:: @@ -102,7 +103,7 @@ where ``/path/to/hsd`` is the path to this top-level directory (e.g., ``/Users/J NOAA RDHPCS Systems ---------------------- -On many NOAA :term:`RDHPCS`, a container named ``ubuntu22.04-intel-ue-1.6.0-wm-hsd.img`` has already been built, and users may access the container at the locations in :numref:`Table %s `. +On many NOAA :term:`RDHPCS`, a container named ``ubuntu22.04-intel2024.2.0-1-devel-wm-hsd.img`` has already been built, and users may access the container at the locations in :numref:`Table %s `. .. _PreBuiltContainers: @@ -120,36 +121,34 @@ On many NOAA :term:`RDHPCS`, a container named ``ubuntu22.04-intel-ue-1.6.0-wm-h | Orion/Hercules | /work/noaa/epic/role-epic/contrib/containers | +--------------------+--------------------------------------------------------+ -.. [#fn] The CAPE case can run on the NOAA Cloud ParallelWorks (PW) platforms, but the baroclinic wave and tropical cyclone cases cannot. - Users can simply set an environment variable to point to the container: .. code-block:: console - export img=path/to/ubuntu22.04-intel-ue-1.6.0-wm-hsd.img + export img=path/to/ubuntu22.04-intel2024.2.0-1-devel-wm-hsd.img If users prefer, they may copy the container to their local working directory. For example, on Gaea .. code-block:: console - cp /gpfs/f6/bil-fire8/world-shared/containers/ubuntu22.04-intel-ue-1.6.0-wm-hsd.img . + cp /gpfs/f6/bil-fire8/world-shared/containers/ubuntu22.04-intel2024.2.0-1-devel-wm-hsd.img . Other Systems ---------------- -On other systems, users can build the Singularity container from a public Docker :term:`container` image or download the ``ubuntu22.04-intel-ue-1.6.0-wm-hsd.img`` container from the `UFS Hierarchical Testing Framework (HTF) Data Bucket `_. Downloading may be faster depending on the download speed on the user's system. Note that the container in the data bucket is from the May 30, 2025 ``develop`` branch. +On other systems, users can build the Singularity container from a public Docker :term:`container` image or download the ``ubuntu22.04-intel2024.2.0-1-devel-wm-hsd.img`` container from the `UFS Hierarchical Testing Framework (HTF) Data Bucket `_. Downloading may be faster depending on the download speed on the user's system. Note that the container in the data bucket is from the May 30, 2025 ``develop`` branch. To download from the data bucket, users can run: .. code-block:: console - wget https://noaa-ufs-htf-pds.s3.amazonaws.com/develop-20250530/ubuntu22.04-intel-ue-1.6.0-wm-hsd.img + wget https://noaa-ufs-htf-pds.s3.amazonaws.com/develop-20260212/ubuntu22.04-intel2024.2.0-1-devel-wm-hsd.img To build the container from a Docker image, users can run: .. code-block:: console - singularity build --force ubuntu22.04-intel-ue-1.6.0-wm-hsd.img docker://noaaepic/ubuntu22.04-intel2023.2.1-wm:ue160-fms202401-dev-hsd + singularity build --force ubuntu22.04-intel2024.2.0-1-devel-wm-hsd.img docker://noaaepic/ubuntu22.04-intel2024.2.0-1-devel-wm:ue192-hsd-aquaplanet This process may take several hours depending on the system. @@ -169,7 +168,7 @@ Users on any system may download and untar the data from the `UFS Hierarchical T .. code-block:: console cd $HSD - wget https://noaa-ufs-htf-pds.s3.amazonaws.com/develop-20250530/HSD_fix_files_and_case_data.tar.gz + wget https://noaa-ufs-htf-pds.s3.amazonaws.com/develop-20260212/HSD_fix_files_and_case_data.tar.gz tar xvfz HSD_fix_files_and_case_data.tar.gz .. _RunContainer: @@ -192,7 +191,7 @@ Save the location of the container in an environment variable. .. code-block:: console - export img=/path/to/ubuntu22.04-intel-ue-1.6.0-wm-hsd.img + export img=/path/to/ubuntu22.04-intel2024.2.0-1-devel-wm-hsd.img Users may convert a container ``.img`` file to a writable sandbox. This step is optional and unnecessary on most systems (it can take several hours): @@ -205,7 +204,7 @@ When making a writable sandbox on NOAA :term:`RDHPCS`, the following warnings co .. code-block:: console INFO: Starting build... - INFO: Verifying bootstrap image ubuntu22.04-intel-ue-1.6.0-wm-hsd.img + INFO: Verifying bootstrap image ubuntu22.04-intel2024.2.0-1-devel-wm-hsd.img WARNING: integrity: signature not found for object group 1 WARNING: Bootstrap image could not be verified, but build will continue. @@ -235,10 +234,10 @@ Run the ``stage-rt.sh`` script with the proper arguments. where: - * ``-c`` is the compiler on the user's local machine (e.g., ``intel/2022.1.2``, ``intel-oneapi-compilers/2022.2.1``, ``intel/2023.2.0``) - * ``-m`` is the :term:`MPI` on the user's local machine (e.g., ``impi/2022.1.2``, ``intel-oneapi-mpi/2021.7.1``, ``cray-mpich/8.1.28``) - * ``-p`` refers to the local machine/platform (e.g., ``ursa``, ``gaea``, ``noaacloud``). Required for Gaea, Hercules, and Orion only. - * ``-i`` is the full path to the container image (e.g., ``$img`` or ``$HSD/ubuntu22.04-intel-ue-1.6.0-wm-hsd.img``). + * ``-c`` is the compiler on the user's local machine (e.g., ``intel/2024.2.1``, ``intel-oneapi-compilers/2024.1.0``, ``intel-oneapi-compilers/2024.2.1``) + * ``-m`` is the :term:`MPI` on the user's local machine (e.g., ``impi/2024.2.1``, ``intel-oneapi-mpi/2021.12.0``, ``intel-oneapi-mpi/2021.13.1``) + * ``-p`` refers to the local machine/platform (e.g., ``ursa``, ``gaea``, ``noaacloud``). Required for Gaea, Hercules, Orion, and Ursa only. + * ``-i`` is the full path to the container image (e.g., ``$img`` or ``$HSD/ubuntu22.04-intel2024.2.0-1-devel-wm-hsd.img``). .. note:: @@ -263,7 +262,7 @@ Additionally, the user should see the ``ufs-weather-model`` directory in the ``$ .. note:: Gaea: - * Gaea uses a different compiler and MPI to run with the container: ``-c=intel-classic/2023.2.0 -m=cray-mpich/8.1.28`` + * Gaea uses a different compiler and MPI to run with the container: ``-c=intel-oneapi/2024.2 -m=cray-mpich/8.1.32`` .. _ConfigureExptC: @@ -285,10 +284,11 @@ Default variables for regression tests and HSD tests are set in the ``default_va Test Configuration -------------------- -Additional configuration may be needed for the specific test the user plans to run. For information on test-specific configuration, view the information for specific tests: +Additional configuration may be needed for the specific test the user plans to run. For information on test-specific configuration, view the information for specific tests: * The :ref:`July 2020 CAPE Test Configuration ` * The :ref:`Baroclinic Instability Test Configuration ` + * The :ref:`AquaPlanet Test Configuration ` * The :ref:`Idealized, Regional Tropical Cyclone Test Configuration ` .. _RunExptC: @@ -306,7 +306,7 @@ To start the experiment, run: where: * ````: Account/project number for batch jobs. -* ````: Name of the test case (e.g., ``2020_CAPE``, ``baroclinic_wave``, or ``tropical_cyclone``). +* ````: Name of the test case (e.g., ``2020_CAPE``, ``baroclinic_wave``, ``aquaplanet``, or ``tropical_cyclone``). * ````: Compiler used for the tests (``intel`` or ``gnu``). The script will loop until it runs both tasks or crashes. ``rococtostat`` can be used to track its progress; see the :ref:`Track Progress ` section for details. @@ -360,7 +360,7 @@ If the experiment completes successfully, the loop will exit with output similar + TEST_END_TIME='20241115 16:43:41' + export TEST_END_TIME + python -c 'import create_log; create_log.finish_log()' - running: /usr/bin/singularity exec --env-file /scratch1/NCEPDEV/stmp4/User.Name/hsd-test/new-cont/ufs-weather-model/container-scripts/ufswm.env -B /scratch1:/scratch1 /scratch1/NCEPDEV/stmp4/User.Name/hsd-test/new-cont/ubuntu22.04-intel-ue-1.6.0-wm-hsd.img python tmp_arg_file.py + running: /usr/bin/singularity exec --env-file /scratch1/NCEPDEV/stmp4/User.Name/hsd-test/new-cont/ufs-weather-model/container-scripts/ufswm.env -B /scratch1:/scratch1 /scratch1/NCEPDEV/stmp4/User.Name/hsd-test/new-cont/ubuntu22.04-intel2024.2.0-1-devel-wm-hsd.img python tmp_arg_file.py Performing Cleanup... REGRESSION TEST RESULT: SUCCESS + echo 'ufs_test.sh finished' From e14cc477c06cb94d2c152478e4e759e5a6a5d9eb Mon Sep 17 00:00:00 2001 From: Joshua Kublnick Date: Fri, 13 Feb 2026 15:21:36 -0500 Subject: [PATCH 09/32] Sea-Land mask data and setting up atompsheric profiles section --- doc/UsersGuide/source/AquaPlanet.rst | 68 +++++++++++++++++++++++++++- 1 file changed, 67 insertions(+), 1 deletion(-) diff --git a/doc/UsersGuide/source/AquaPlanet.rst b/doc/UsersGuide/source/AquaPlanet.rst index 4e93853c47..0478bd6874 100644 --- a/doc/UsersGuide/source/AquaPlanet.rst +++ b/doc/UsersGuide/source/AquaPlanet.rst @@ -384,4 +384,70 @@ Copy the modified file back: .. code-block:: console - cp new-IMS-NIC.blended.ice.monthly.clim.grb /IMS-NIC.blended.ice.monthly.clim.grb \ No newline at end of file + cp new-IMS-NIC.blended.ice.monthly.clim.grb /IMS-NIC.blended.ice.monthly.clim.grb + +Edit Sea-Land Mask Data +----------------------- + +Navigate to the sea-land mask tool directory: + +.. code-block:: console + + cd ~/Aquaplanet/slmask/ + +Copy the sea-land mask file from your run directory: + +.. code-block:: console + + cp /global_slmask.t62.192.94.grb . + +Compile and run the tool: + +.. code-block:: console + + ./compile.sh + ./slmask.x + +Copy the modified file back: + +.. code-block:: console + + cp new-global_slmask.t62.192.94.grb /global_slmask.t62.192.94.grb + +Setup Atmospheric Profiles +--------------------------- + +Navigate to the atmospheric profile tool directory: + +.. code-block:: console + + cd ~/Aquaplanet/atmos-profile + +Copy the atmospheric analysis file from ``$EXTRACT_DIR``: + +.. code-block:: console + + cp $EXTRACT_DIR/gfs.t06z.atmanl.nc . + +Compile and load required modules: + +.. code-block:: console + + ./compile.sh + module purge + module use /contrib/spack-stack/spack-stack-1.9.2/envs/ue-oneapi-2024.2.1/install/modulefiles/Core + module load stack-oneapi/2024.2.1 + module load stack-intel-oneapi-mpi/2021.13 + module load netcdf-fortran/4.6.1 + +Run the tool: + +.. code-block:: console + + ./profile.x + +Copy the modified atmospheric file back to the extract directory: + +.. code-block:: console + + cp gfs.t06z.atmanl.nc $EXTRACT_DIR/gfs.20251015/06/atmos/gfs.t06z.atmanl.nc \ No newline at end of file From a3784da191eb34d82e9f0e96005c8d5159bcafa8 Mon Sep 17 00:00:00 2001 From: Joshua Kublnick Date: Fri, 13 Feb 2026 15:28:32 -0500 Subject: [PATCH 10/32] Orography field and regen ICs sections --- doc/UsersGuide/source/AquaPlanet.rst | 47 +++++++++++++++++++++++++++- 1 file changed, 46 insertions(+), 1 deletion(-) diff --git a/doc/UsersGuide/source/AquaPlanet.rst b/doc/UsersGuide/source/AquaPlanet.rst index 0478bd6874..84c8ceed32 100644 --- a/doc/UsersGuide/source/AquaPlanet.rst +++ b/doc/UsersGuide/source/AquaPlanet.rst @@ -450,4 +450,49 @@ Copy the modified atmospheric file back to the extract directory: .. code-block:: console - cp gfs.t06z.atmanl.nc $EXTRACT_DIR/gfs.20251015/06/atmos/gfs.t06z.atmanl.nc \ No newline at end of file + cp gfs.t06z.atmanl.nc $EXTRACT_DIR/gfs.20251015/06/atmos/gfs.t06z.atmanl.nc + +Prepare Orography Fields +------------------------- + +Navigate to the orography profile tool directory: + +.. code-block:: console + + cd ~/Aquaplanet/orog-profile/ + +Compile the tool: + +.. code-block:: console + + ./compile.sh + +Verify that ``list.txt`` contains the correct relative path to fix orography files, then run: + +.. code-block:: console + + ./patch.sh + +This will modify the orography files in place to set land mask to ocean and height to zero. + +Regenerate Initial Conditions +------------------------------ + +Return to the ``gdas_init`` utility directory: + +.. code-block:: console + + cd ~/UFS_UTILS/util/gdas_init/ + +.. important:: + + Set ``EXTRACT_DATA=no`` in the ``config`` file since data is already extracted. + +Delete or rename the old output directory, then run the driver script: + +.. code-block:: console + + mv $OUTDIR ${OUTDIR}.old + ./driver..sh + +Check the results in the output directory. \ No newline at end of file From ac058508ae295556f66103f4613ad64bcc9e31c1 Mon Sep 17 00:00:00 2001 From: Joshua Kublnick Date: Fri, 13 Feb 2026 15:34:34 -0500 Subject: [PATCH 11/32] Setup surface profiles docs --- doc/UsersGuide/source/AquaPlanet.rst | 65 +++++++++++++++++++++++++--- 1 file changed, 60 insertions(+), 5 deletions(-) diff --git a/doc/UsersGuide/source/AquaPlanet.rst b/doc/UsersGuide/source/AquaPlanet.rst index 84c8ceed32..0fe9fe3586 100644 --- a/doc/UsersGuide/source/AquaPlanet.rst +++ b/doc/UsersGuide/source/AquaPlanet.rst @@ -452,6 +452,44 @@ Copy the modified atmospheric file back to the extract directory: cp gfs.t06z.atmanl.nc $EXTRACT_DIR/gfs.20251015/06/atmos/gfs.t06z.atmanl.nc +Setup Surface Profiles +----------------------- + +Navigate to the surface profile tool directory: + +.. code-block:: console + + cd ~/Aquaplanet/sfc-profile/ + +Copy the surface analysis file from ``$EXTRACT_DIR``: + +.. code-block:: console + + cp $EXTRACT_DIR/gfs.t06z.sfcanl.nc . + +Compile and load required modules: + +.. code-block:: console + + ./compile.sh + module purge + module use /contrib/spack-stack/spack-stack-1.9.2/envs/ue-oneapi-2024.2.1/install/modulefiles/Core + module load stack-oneapi/2024.2.1 + module load stack-intel-oneapi-mpi/2021.13 + module load netcdf-fortran/4.6.1 + +Run the tool: + +.. code-block:: console + + ./profile.x + +Copy the modified surface file back to the extract directory: + +.. code-block:: console + + cp gfs.t06z.sfcanl.nc $EXTRACT_DIR/gfs.20251015/06/atmos/gfs.t06z.sfcanl.nc + Prepare Orography Fields ------------------------- @@ -478,7 +516,9 @@ This will modify the orography files in place to set land mask to ocean and heig Regenerate Initial Conditions ------------------------------ -Return to the ``gdas_init`` utility directory: +Now that the correct fixed files are in place, return to UFS_UTILS and run ``chgres`` again. + +Navigate back to the ``gdas_init`` utility directory: .. code-block:: console @@ -486,13 +526,28 @@ Return to the ``gdas_init`` utility directory: .. important:: - Set ``EXTRACT_DATA=no`` in the ``config`` file since data is already extracted. + In the ``config`` file, make sure that ``EXTRACT_DATA`` is set to ``no`` since data is already extracted. + +Delete or rename the output directory. For example: -Delete or rename the old output directory, then run the driver script: +.. code-block:: console + + rm -rf $OUTDIR + +or: + +.. code-block:: console + + mv $OUTDIR $OUTDIR.backup + +Run the driver script: .. code-block:: console - mv $OUTDIR ${OUTDIR}.old ./driver..sh -Check the results in the output directory. \ No newline at end of file +Check the results in: + +.. code-block:: console + + ../../output/gfs.20251015/06/model/atmos/input/ \ No newline at end of file From 8327f8bffec06b7b9a62aae6a38c7dce65a62f2d Mon Sep 17 00:00:00 2001 From: Joshua Kublnick Date: Fri, 13 Feb 2026 15:42:02 -0500 Subject: [PATCH 12/32] Modifying source code, copying ICs, adding Noah-MP variables. (docs) --- doc/UsersGuide/source/AquaPlanet.rst | 79 +++++++++++++++++++++++++++- 1 file changed, 78 insertions(+), 1 deletion(-) diff --git a/doc/UsersGuide/source/AquaPlanet.rst b/doc/UsersGuide/source/AquaPlanet.rst index 0fe9fe3586..efa0b9b2db 100644 --- a/doc/UsersGuide/source/AquaPlanet.rst +++ b/doc/UsersGuide/source/AquaPlanet.rst @@ -550,4 +550,81 @@ Check the results in: .. code-block:: console - ../../output/gfs.20251015/06/model/atmos/input/ \ No newline at end of file + ../../output/gfs.20251015/06/model/atmos/input/ + +Add Noah-MP Variables to Surface Files +--------------------------------------- + +Since the GFS_v17 physics suite is used, empty fields (``sheleg``, ``snwdph``, and ``zorl``) must be added to the surface files: + +.. code-block:: console + + cd ~/Aquaplanet/noah-MP-vars/ + +Copy the surface files from the output directory: + +.. code-block:: console + + cp $OUTDIR/gfs.20251015/06/model/atmos/input/sfc_* . + +Compile and run: + +.. code-block:: console + + ./compile.sh + ./add-vars.x + +Copy the modified files back: + +.. code-block:: console + + cp sfc_* $OUTDIR/gfs.20251015/06/model/atmos/input/. + +Copy Initial Conditions to Run Directory +----------------------------------------- + +Copy all generated initial condition files to your run directory: + +.. code-block:: console + + cp $OUTDIR/gfs.20251015/06/model/atmos/input/* /INPUT/. + +Copy the modified orography files: + +.. code-block:: console + + cd /INPUT/ + cp ~/UFS_UTILS/fix/orog/C48/C48.mx500_oro_data.tile1.nc oro_data.tile1.nc + cp ~/UFS_UTILS/fix/orog/C48/C48.mx500_oro_data.tile2.nc oro_data.tile2.nc + cp ~/UFS_UTILS/fix/orog/C48/C48.mx500_oro_data.tile3.nc oro_data.tile3.nc + cp ~/UFS_UTILS/fix/orog/C48/C48.mx500_oro_data.tile4.nc oro_data.tile4.nc + cp ~/UFS_UTILS/fix/orog/C48/C48.mx500_oro_data.tile5.nc oro_data.tile5.nc + cp ~/UFS_UTILS/fix/orog/C48/C48.mx500_oro_data.tile6.nc oro_data.tile6.nc + +Modify Model Source Code +------------------------- + +Two minor modifications are needed to the UFS Weather Model source code for the aquaplanet configuration. + +**Fix Solar Zenith Angle** + +Edit the file ``./ufs-weather-model/UFSATM/ccpp/physics/physics/Radiation/radiation_astronomy.f``. After line 600, add: + +.. code-block:: console + + ! AQUAPLANET TEST CASE + solcon=1370. + dlt=0. + ! AQUAPLANET TEST CASE + +**Reduce Land-Surface Warnings** + +Edit the file ``./ufs-weather-model/UFSATM/fv3/atmos_cubed_sphere/tools/fv_diagnostics.F90``. After lines 4292 and 4349, add: + +.. code-block:: console + + ! AQUAPLANET TEST CASE + return + ! AQUAPLANET TEST CASE + +Recompile the model after making these changes and update the executable in your run directory. \ No newline at end of file From ede598580422a2da08f90a0152841f3a410f0702 Mon Sep 17 00:00:00 2001 From: Joshua Kublnick Date: Fri, 13 Feb 2026 15:50:38 -0500 Subject: [PATCH 13/32] (Adding 90-day spinup section --- doc/UsersGuide/source/AquaPlanet.rst | 97 +++++++++++++++++++++++++++- 1 file changed, 96 insertions(+), 1 deletion(-) diff --git a/doc/UsersGuide/source/AquaPlanet.rst b/doc/UsersGuide/source/AquaPlanet.rst index efa0b9b2db..0e5669dfd5 100644 --- a/doc/UsersGuide/source/AquaPlanet.rst +++ b/doc/UsersGuide/source/AquaPlanet.rst @@ -627,4 +627,99 @@ Edit the file ``./ufs-weather-model/UFSATM/fv3/atmos_cubed_sphere/tools/fv_diagn return ! AQUAPLANET TEST CASE -Recompile the model after making these changes and update the executable in your run directory. \ No newline at end of file +Recompile the model after making these changes and update the executable in your run directory. + +Running the 90-Day Spin-Up +--------------------------- + +The aquaplanet simulation requires a 90-day spin-up period to reach equilibrium. This is run in three 30-day (720-hour) segments with restarts. + +**Initial 30-Day Run** + +Edit the ``job_card`` file in your run directory: + +.. code-block:: console + + #SBATCH --time=480 + +Edit the ``model_configure`` file: + +.. code-block:: console + + start_year: 2025 + start_month: 10 + start_day: 15 + nhours_fcst: 720 + quilting_restart: .false. + output_fh: 24 -1 + +Submit the job: + +.. code-block:: console + + sbatch job_card + +**First Restart (Days 31-60)** + +After the first month completes, prepare for the restart: + +.. code-block:: console + + cd RESTART/ + for file in 20*.060000.*.nc; do mv "$file" "${file#20*.060000.}"; done + mv 20*.060000.coupler.res coupler.res + mv * ../INPUT/. + +Edit ``input.nml`` and set: + +.. code-block:: console + + make_nh = .false. + na_init = 0 + external_ic = .false. + nggps_ic = .false. + mountain = .true. + warm_start = .true. + +Edit ``model_configure`` and set: + +.. code-block:: console + + nhours_fcst: 1440 + fhrot: 720 + +Submit the job: + +.. code-block:: console + + sbatch job_card + +**Second Restart (Days 61-90)** + +After the second month completes: + +.. code-block:: console + + cd RESTART/ + for file in 20*.060000.*.nc; do mv "$file" "${file#20*.060000.}"; done + mv 20*.060000.coupler.res coupler.res + mv * ../INPUT/. + +Keep ``input.nml`` settings from the first restart. Edit ``model_configure`` and set: + +.. code-block:: console + + nhours_fcst: 2160 + fhrot: 1440 + +Submit the job: + +.. code-block:: console + + sbatch job_card + +After this final segment completes, the files in the ``RESTART`` directory represent spun-up initial conditions. Rename and copy these files to the ``INPUT`` directory --- these are the new initial conditions for your experiment (spun up for 90 days). Save them for future use. + +To continue running experiments with these self-generated initial conditions, follow the instructions in :numref:`Section %s `, using the ``input.nml`` from the restart runs and updating ``fhrot`` and ``nhours_fcst`` in ``model_configure`` as described in the extended experiments section. + +.. include:: ./doc-snippets/hsd_notes.rst \ No newline at end of file From 6feeb9fcb20c2712bb5ee78061fc990cbfa29712 Mon Sep 17 00:00:00 2001 From: Joshua Kublnick Date: Fri, 13 Feb 2026 16:05:20 -0500 Subject: [PATCH 14/32] Quick fix --- doc/UsersGuide/source/AquaPlanet.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/UsersGuide/source/AquaPlanet.rst b/doc/UsersGuide/source/AquaPlanet.rst index 0e5669dfd5..b2e523a329 100644 --- a/doc/UsersGuide/source/AquaPlanet.rst +++ b/doc/UsersGuide/source/AquaPlanet.rst @@ -112,7 +112,7 @@ For example, to monitor progress or check results for the ``aquaplanet`` case, r tail -f ${UFS_WM}/tests-dev/run_dir/aquaplanet_intel/err tail -f ${UFS_WM}/tests-dev/run_dir/aquaplanet_intel/out - .. _plotting-aquaplanet: +.. _plotting-aquaplanet: ======================= Plotting Script From 4b51d77718fead505b1cfa2e339b095cdb721f92 Mon Sep 17 00:00:00 2001 From: Gillian Petro Date: Tue, 17 Feb 2026 15:57:35 -0500 Subject: [PATCH 15/32] minor updates --- doc/UsersGuide/source/AquaPlanet.rst | 16 ++++++++-------- doc/UsersGuide/source/BuildingAndRunning.rst | 2 -- doc/UsersGuide/source/HSD.rst | 2 ++ 3 files changed, 10 insertions(+), 10 deletions(-) diff --git a/doc/UsersGuide/source/AquaPlanet.rst b/doc/UsersGuide/source/AquaPlanet.rst index b2e523a329..07c0d0f43f 100644 --- a/doc/UsersGuide/source/AquaPlanet.rst +++ b/doc/UsersGuide/source/AquaPlanet.rst @@ -4,10 +4,10 @@ .. _aquaplanet: ********************* -AquaPlanet Test Case +Aquaplanet Test Case ********************* -The AquaPlanet test case is an idealized atmosphere-only forecast configuration designed to study atmospheric dynamics in a simplified Earth-like setting where all land is replaced with ocean. This configuration removes the complexity of land-surface interactions, topography, and regional variations, allowing researchers to focus on fundamental atmospheric processes such as tropical convection, Hadley circulation, jet stream dynamics, and the global energy budget. +The Aquaplanet test case is an idealized atmosphere-only forecast configuration designed to study atmospheric dynamics in a simplified Earth-like setting where all land is replaced with ocean. This configuration removes the complexity of land-surface interactions, topography, and regional variations, allowing researchers to focus on fundamental atmospheric processes such as tropical convection, Hadley circulation, jet stream dynamics, and the global energy budget. The test case runs at C48 resolution with the ``FV3_GFS_v17_p8_ugwpv1`` physics suite. Initial conditions are created by modifying standard GFS data to represent an aquaplanet configuration: sea surface temperatures (SST) follow an idealized latitudinal profile, all land is converted to ocean, topography is set to zero, and sea ice is removed. The atmospheric initial state is configured with idealized vertical profiles of temperature, humidity, and winds appropriate for an aquaplanet simulation. @@ -22,10 +22,10 @@ Obtaining Data for HSD Cases .. _run-aquaplanet: ============================== -Running the AquaPlanet Case +Running the Aquaplanet Case ============================== -This section explains how to run the AquaPlanet case using the ``ufs_test.sh`` script with pre-staged initial conditions. This is the recommended way to run the case for most users. +This section explains how to run the Aquaplanet case using the ``ufs_test.sh`` script with pre-staged initial conditions. This is the recommended way to run the case for most users. Clone the Repository -------------------- @@ -157,13 +157,13 @@ To use the plotting script with user-generated data: .. _setup-aquaplanet: -========================================================== -Advanced: Setting Up the AquaPlanet Experiment from Scratch -========================================================== +============================================================ +Advanced: Setting Up the Aquaplanet Experiment from Scratch +============================================================ .. note:: - This section is **optional**. Most users can run the AquaPlanet case using the ``ufs_test.sh`` method described above, which uses pre-staged initial conditions. The steps below are for advanced users who wish to create their own initial conditions from scratch. + This section is **optional**. Most users can run the Aquaplanet case using the ``ufs_test.sh`` method described above, which uses pre-staged initial conditions. The steps below are for advanced users who wish to create their own initial conditions from scratch. The from-scratch setup involves editing orography, SST, ice, and sea-land mask files to create an aquaplanet configuration, then running a 90-day spin-up to allow the model to reach a balanced state. This process produces the same initial conditions that are provided pre-staged for the standard test case. diff --git a/doc/UsersGuide/source/BuildingAndRunning.rst b/doc/UsersGuide/source/BuildingAndRunning.rst index 250d3df989..4cab058c0b 100644 --- a/doc/UsersGuide/source/BuildingAndRunning.rst +++ b/doc/UsersGuide/source/BuildingAndRunning.rst @@ -310,8 +310,6 @@ the defaults. - Set default values for the coupled 5-component tests using CMEPS. * - export_cpl - Set default values for *coupled* / S2S configurations. - * - export_35d_run - - Set default values for EMC's weekly coupled benchmark 35d tests (see `rt_35d.conf `__). * - export_datm_cdeps - Set default values for configurations that use the data atmosphere (:term:`DATM`) component. * - export_hafs_datm_cdeps diff --git a/doc/UsersGuide/source/HSD.rst b/doc/UsersGuide/source/HSD.rst index 4017bb1b90..c14936e364 100644 --- a/doc/UsersGuide/source/HSD.rst +++ b/doc/UsersGuide/source/HSD.rst @@ -17,6 +17,7 @@ Although the UFS Weather Model (WM) can be run in any of several configurations, CAPE2020 baroclinic_wave IdealizedTC + Aquaplanet AddingHSDCase HSDcontainer @@ -25,6 +26,7 @@ Currently, users can find information on running the following HSD cases: * The :ref:`July 2020 CAPE Case ` * The :ref:`Baroclinic Instability Case ` * The :ref:`Idealized, Regional Tropical Cyclone Case ` + * The :ref:`Aquaplanet Case ` For a full list of supported WM configurations, view the `rt.conf `_ file. From 121f1f28012ee56162fb7f202d54c8bd4890363f Mon Sep 17 00:00:00 2001 From: Gillian Petro Date: Tue, 17 Feb 2026 16:19:59 -0500 Subject: [PATCH 16/32] Eddie's suggestions --- doc/UsersGuide/source/HSDcontainer.rst | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/doc/UsersGuide/source/HSDcontainer.rst b/doc/UsersGuide/source/HSDcontainer.rst index 7518ab4c3b..c474dbab01 100644 --- a/doc/UsersGuide/source/HSDcontainer.rst +++ b/doc/UsersGuide/source/HSDcontainer.rst @@ -103,7 +103,7 @@ where ``/path/to/hsd`` is the path to this top-level directory (e.g., ``/Users/J NOAA RDHPCS Systems ---------------------- -On many NOAA :term:`RDHPCS`, a container named ``ubuntu22.04-intel2024.2.0-1-devel-wm-hsd.img`` has already been built, and users may access the container at the locations in :numref:`Table %s `. +On many NOAA :term:`RDHPCS`, a container named ``ubuntu22.04-intel-ue-1.9.2-wm-hsd.img`` has already been built, and users may access the container at the locations in :numref:`Table %s `. .. _PreBuiltContainers: @@ -116,7 +116,7 @@ On many NOAA :term:`RDHPCS`, a container named ``ubuntu22.04-intel2024.2.0-1-dev +--------------------+--------------------------------------------------------+ | Ursa | /scratch3/NCEPDEV/nems/role.epic/containers | +--------------------+--------------------------------------------------------+ - | NOAA Cloud [#fn]_ | /contrib/EPIC/containers | + | NOAA Cloud | /contrib/EPIC/containers | +--------------------+--------------------------------------------------------+ | Orion/Hercules | /work/noaa/epic/role-epic/contrib/containers | +--------------------+--------------------------------------------------------+ @@ -125,30 +125,30 @@ Users can simply set an environment variable to point to the container: .. code-block:: console - export img=path/to/ubuntu22.04-intel2024.2.0-1-devel-wm-hsd.img + export img=path/to/ubuntu22.04-intel-ue-1.9.2-wm-hsd.img If users prefer, they may copy the container to their local working directory. For example, on Gaea .. code-block:: console - cp /gpfs/f6/bil-fire8/world-shared/containers/ubuntu22.04-intel2024.2.0-1-devel-wm-hsd.img . + cp /gpfs/f6/bil-fire8/world-shared/containers/ubuntu22.04-intel-ue-1.9.2-wm-hsd.img . Other Systems ---------------- -On other systems, users can build the Singularity container from a public Docker :term:`container` image or download the ``ubuntu22.04-intel2024.2.0-1-devel-wm-hsd.img`` container from the `UFS Hierarchical Testing Framework (HTF) Data Bucket `_. Downloading may be faster depending on the download speed on the user's system. Note that the container in the data bucket is from the May 30, 2025 ``develop`` branch. +On other systems, users can build the Singularity container from a public Docker :term:`container` image or download the ``ubuntu22.04-intel-ue-1.9.2-wm-hsd.img`` container from the `UFS Hierarchical Testing Framework (HTF) Data Bucket `_. Downloading may be faster depending on the download speed on the user's system. Note that the container in the data bucket is from the May 30, 2025 ``develop`` branch. To download from the data bucket, users can run: .. code-block:: console - wget https://noaa-ufs-htf-pds.s3.amazonaws.com/develop-20260212/ubuntu22.04-intel2024.2.0-1-devel-wm-hsd.img + wget https://noaa-ufs-htf-pds.s3.amazonaws.com/develop-20260212/ubuntu22.04-intel-ue-1.9.2-wm-hsd.img To build the container from a Docker image, users can run: .. code-block:: console - singularity build --force ubuntu22.04-intel2024.2.0-1-devel-wm-hsd.img docker://noaaepic/ubuntu22.04-intel2024.2.0-1-devel-wm:ue192-hsd-aquaplanet + singularity build --force ubuntu22.04-intel-ue-1.9.2-wm-hsd.img docker://noaaepic/ubuntu22.04-intel-ue-1.9.2-wm-hsd.img:ue192-hsd-aquaplanet This process may take several hours depending on the system. @@ -191,20 +191,20 @@ Save the location of the container in an environment variable. .. code-block:: console - export img=/path/to/ubuntu22.04-intel2024.2.0-1-devel-wm-hsd.img + export img=/path/to/ubuntu22.04-intel-ue-1.9.2-wm-hsd.img Users may convert a container ``.img`` file to a writable sandbox. This step is optional and unnecessary on most systems (it can take several hours): .. code-block:: console - singularity build --sandbox ubuntu22.04-intel-ue-1.6.0-wm-hsd $img + singularity build --sandbox ubuntu22.04-intel-ue-1.9.2-wm-hsd $img When making a writable sandbox on NOAA :term:`RDHPCS`, the following warnings commonly appear and can be ignored: .. code-block:: console INFO: Starting build... - INFO: Verifying bootstrap image ubuntu22.04-intel2024.2.0-1-devel-wm-hsd.img + INFO: Verifying bootstrap image ubuntu22.04-intel-ue-1.9.2-wm-hsd.img WARNING: integrity: signature not found for object group 1 WARNING: Bootstrap image could not be verified, but build will continue. @@ -237,7 +237,7 @@ where: * ``-c`` is the compiler on the user's local machine (e.g., ``intel/2024.2.1``, ``intel-oneapi-compilers/2024.1.0``, ``intel-oneapi-compilers/2024.2.1``) * ``-m`` is the :term:`MPI` on the user's local machine (e.g., ``impi/2024.2.1``, ``intel-oneapi-mpi/2021.12.0``, ``intel-oneapi-mpi/2021.13.1``) * ``-p`` refers to the local machine/platform (e.g., ``ursa``, ``gaea``, ``noaacloud``). Required for Gaea, Hercules, Orion, and Ursa only. - * ``-i`` is the full path to the container image (e.g., ``$img`` or ``$HSD/ubuntu22.04-intel2024.2.0-1-devel-wm-hsd.img``). + * ``-i`` is the full path to the container image (e.g., ``$img`` or ``$HSD/ubuntu22.04-intel-ue-1.9.2-wm-hsd.img``). .. note:: @@ -360,7 +360,7 @@ If the experiment completes successfully, the loop will exit with output similar + TEST_END_TIME='20241115 16:43:41' + export TEST_END_TIME + python -c 'import create_log; create_log.finish_log()' - running: /usr/bin/singularity exec --env-file /scratch1/NCEPDEV/stmp4/User.Name/hsd-test/new-cont/ufs-weather-model/container-scripts/ufswm.env -B /scratch1:/scratch1 /scratch1/NCEPDEV/stmp4/User.Name/hsd-test/new-cont/ubuntu22.04-intel2024.2.0-1-devel-wm-hsd.img python tmp_arg_file.py + running: /usr/bin/singularity exec --env-file /scratch1/NCEPDEV/stmp4/User.Name/hsd-test/new-cont/ufs-weather-model/container-scripts/ufswm.env -B /scratch1:/scratch1 /scratch1/NCEPDEV/stmp4/User.Name/hsd-test/new-cont/ubuntu22.04-intel-ue-1.9.2-wm-hsd.img python tmp_arg_file.py Performing Cleanup... REGRESSION TEST RESULT: SUCCESS + echo 'ufs_test.sh finished' From a3f3450548dc3fb8f50ce593b53eae9ef3f1d768 Mon Sep 17 00:00:00 2001 From: Gillian Petro Date: Wed, 18 Feb 2026 00:03:13 -0500 Subject: [PATCH 17/32] add clarifications --- doc/UsersGuide/source/AquaPlanet.rst | 54 +++++++++++-------- .../source/doc-snippets/hsd_run_tests.rst | 2 +- 2 files changed, 33 insertions(+), 23 deletions(-) diff --git a/doc/UsersGuide/source/AquaPlanet.rst b/doc/UsersGuide/source/AquaPlanet.rst index 07c0d0f43f..3f740e856b 100644 --- a/doc/UsersGuide/source/AquaPlanet.rst +++ b/doc/UsersGuide/source/AquaPlanet.rst @@ -9,7 +9,7 @@ Aquaplanet Test Case The Aquaplanet test case is an idealized atmosphere-only forecast configuration designed to study atmospheric dynamics in a simplified Earth-like setting where all land is replaced with ocean. This configuration removes the complexity of land-surface interactions, topography, and regional variations, allowing researchers to focus on fundamental atmospheric processes such as tropical convection, Hadley circulation, jet stream dynamics, and the global energy budget. -The test case runs at C48 resolution with the ``FV3_GFS_v17_p8_ugwpv1`` physics suite. Initial conditions are created by modifying standard GFS data to represent an aquaplanet configuration: sea surface temperatures (SST) follow an idealized latitudinal profile, all land is converted to ocean, topography is set to zero, and sea ice is removed. The atmospheric initial state is configured with idealized vertical profiles of temperature, humidity, and winds appropriate for an aquaplanet simulation. +The test case runs at C48 resolution with the ``FV3_GFS_v17_p8_ugwpv1`` physics suite. Initial conditions are created by modifying standard GFS data to represent an aquaplanet configuration: Sea surface temperatures (SST) follow an idealized latitudinal profile, all land is converted to ocean, topography is set to zero, and sea ice is removed. The atmospheric initial state is configured with idealized vertical profiles of temperature, humidity, and winds appropriate for an aquaplanet simulation. A key feature of this configuration is the 90-day spin-up period required to allow the model to adjust from its initial state to a balanced aquaplanet climate. After spin-up, the simulation can be run for extended periods to study seasonal variations, climate statistics, and atmospheric phenomena in the absence of land-surface influences. @@ -51,45 +51,41 @@ Users with access to the ``epic`` account can run the ``aquaplanet`` test case w ./ufs_test.sh -a epic -s -c -k -r -n "aquaplanet intel" -Where: +where: -- ``-s``: use tests-dev, symlink sharable test scripts -- ``-c``: prevents scripts from comparing with previous results -- ``-k``: keep run directory -- ``-r``: use rocoto scheduler (``-e`` will use ecFlow) -- ``-n "aquaplanet intel"``: use the aquaplanet test case with intel compiler +- ``-s``: uses tests-dev; symlinks sharable test scripts +- ``-c``: prevents scripts from comparing with previous results (by creating new baselines) +- ``-k``: keeps run directory +- ``-r``: uses rocoto scheduler (``-e`` will use ecFlow) +- ``-n "aquaplanet intel"``: uses the aquaplanet test case with intel compiler Running Extended Experiments ----------------------------- After running the base test case, users can extend the simulation by running multiple restart segments. For example, to run a 1-year experiment broken into four 3-month segments: -Navigate to the run directory (linked as ``./tests-dev/run_dir``). - -Use the ``input.nml`` file configured for restart runs (with ``warm_start = .true.``). - -For each 3-month segment, update ``fhrot`` and ``nhours_fcst`` in ``model_configure``: +Navigate to the run directory (``${UFS_WM}/tests-dev/run_dir``). For each 3-month segment, update ``fhrot`` and ``nhours_fcst`` in ``model_configure``: **First segment (months 4-6):** .. code-block:: console - fhrot: 2160 nhours_fcst: 4320 + fhrot: 2160 **Second segment (months 7-9):** .. code-block:: console - fhrot: 4320 nhours_fcst: 6480 + fhrot: 4320 **Third segment (months 10-12):** .. code-block:: console - fhrot: 6480 nhours_fcst: 8640 + fhrot: 6480 After each segment completes, rename and move restart files to the ``INPUT`` directory: @@ -122,7 +118,7 @@ A plotting script is available to generate seasonal mean plots for key atmospher .. code-block:: console - ./tests-dev/test_cases/utils/plot_aq.sh + ${UFS_WM}/tests-dev/test_cases/utils/plot_aq.sh .. note:: @@ -134,11 +130,7 @@ By default, this script creates seasonal means for three variables: - Precipitation patterns - Temperature at 500 mb -The script uses staged data from a 1-year control simulation. If you want to use this dataset on other machines (which have access to :term:`HPSS`), you can retrieve it from HPSS: - -.. code-block:: console - - /5year/NCEPDEV/emc-meso/Ratko.Vasic/AQUAPLANET/1yr-results.tar +The script uses staged data from a 1-year control simulation. If you want to use this dataset on other machines (which have access to :term:`HPSS`), you can retrieve it from HPSS at ``/5year/NCEPDEV/emc-meso/Ratko.Vasic/AQUAPLANET/1yr-results.tar``. Customizing the Plotting Script -------------------------------- @@ -146,15 +138,33 @@ Customizing the Plotting Script To use the plotting script with user-generated data: 1. Copy the script to your run directory. + + .. code-block:: bash + + cd ${UFS_WM}/tests-dev/run_dir/aquaplanet_intel + cp ${UFS_WM}/tests-dev/test_cases/utils/plot_aq.sh . + 2. Edit the variable ``out_pth`` to point to your output location: .. code-block:: bash - out_pth=./ + out_pth=path/to/ufs-weather-model/tests-dev/run_dir/aquaplanet_intel 3. Adjust the seasonal timing variables (``winter_start``, ``spring_start``, etc.), which are given in hours from the start of the 90-day spin-up run (hour 0). + + .. code-block:: bash + + winter_start=2160 + spring_start=4320 + summer_start=6480 + fall_start=8640 + 4. Modify ``season_len`` to set the length of each season in days (typically 90 days, but can be set to 365 for annual means). + .. code-block:: bash + + season_len=90 + .. _setup-aquaplanet: ============================================================ diff --git a/doc/UsersGuide/source/doc-snippets/hsd_run_tests.rst b/doc/UsersGuide/source/doc-snippets/hsd_run_tests.rst index 93d5286894..2802b99f03 100644 --- a/doc/UsersGuide/source/doc-snippets/hsd_run_tests.rst +++ b/doc/UsersGuide/source/doc-snippets/hsd_run_tests.rst @@ -1,4 +1,4 @@ -Launch tests from the ``${UFS_WM}/tests-dev`` directory with the following command: +Users can launch tests from the ``${UFS_WM}/tests-dev`` directory with the following command: .. code-block:: console From cdd8df52d601aaac5e7ae44e2ae739c199174a26 Mon Sep 17 00:00:00 2001 From: Gillian Petro Date: Wed, 18 Feb 2026 00:35:09 -0500 Subject: [PATCH 18/32] adjust clone cmd, minor other fixes --- doc/UsersGuide/source/AquaPlanet.rst | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-) diff --git a/doc/UsersGuide/source/AquaPlanet.rst b/doc/UsersGuide/source/AquaPlanet.rst index 3f740e856b..343bd48a64 100644 --- a/doc/UsersGuide/source/AquaPlanet.rst +++ b/doc/UsersGuide/source/AquaPlanet.rst @@ -193,17 +193,13 @@ Clone the UFS Weather Model: .. code-block:: console - git clone https://github.com/ufs-community/ufs-weather-model.git - cd ufs-weather-model/ - git submodule update --init --recursive + git clone --recursive https://github.com/ufs-community/ufs-weather-model.git Clone UFS_UTILS: .. code-block:: console - git clone https://github.com/ufs-community/UFS_UTILS.git - cd UFS_UTILS/ - git submodule update --init --recursive + git clone --recursive https://github.com/ufs-community/UFS_UTILS.git Clone the aquaplanet tools: @@ -257,7 +253,7 @@ Link the fix directories: cd fix/ ./link_fixdirs.sh emc -Since orography files need to be edited, remove the link and copy the files: +Since orography files need to be edited, remove the link and copy the files. The fix file locations for different platforms are listed in ``link_fixdirs.sh``. For example, on Ursa, users would run: .. code-block:: console From 38b62abbd9bf3bc289f54d61f2d3d93ab41e1171 Mon Sep 17 00:00:00 2001 From: RatkoVasic-NOAA <37597874+RatkoVasic-NOAA@users.noreply.github.com> Date: Wed, 18 Feb 2026 11:24:02 -0500 Subject: [PATCH 19/32] Apply suggestion from @gspetro-NOAA Co-authored-by: Gillian Petro <96886803+gspetro-NOAA@users.noreply.github.com> --- doc/UsersGuide/source/AquaPlanet.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/UsersGuide/source/AquaPlanet.rst b/doc/UsersGuide/source/AquaPlanet.rst index b2e523a329..6038b9b9b1 100644 --- a/doc/UsersGuide/source/AquaPlanet.rst +++ b/doc/UsersGuide/source/AquaPlanet.rst @@ -126,7 +126,7 @@ A plotting script is available to generate seasonal mean plots for key atmospher .. note:: - This plotting script is currently configured to run on Hera and Ursa only. + This plotting script is currently configured to run on Ursa only. By default, this script creates seasonal means for three variables: From eaa521a83227a70058fcdd863f0c1baa8001e4e3 Mon Sep 17 00:00:00 2001 From: Gillian Petro Date: Wed, 18 Feb 2026 12:52:56 -0500 Subject: [PATCH 20/32] clarify UFS_UTILS section --- doc/UsersGuide/source/AquaPlanet.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/doc/UsersGuide/source/AquaPlanet.rst b/doc/UsersGuide/source/AquaPlanet.rst index 343bd48a64..67f1a33a45 100644 --- a/doc/UsersGuide/source/AquaPlanet.rst +++ b/doc/UsersGuide/source/AquaPlanet.rst @@ -263,8 +263,8 @@ Since orography files need to be edited, remove the link and copy the files. The cp /scratch3/NCEPDEV/global/role.glopara/fix/orog/20240917/*.dat orog/. cp -r /scratch3/NCEPDEV/global/role.glopara/fix/orog/20240917/C48/ orog/. -Generate Initial Atmospheric Data ----------------------------------- +Generate Initial Atmospheric Data With UFS_UTILS +-------------------------------------------------- Navigate to the ``gdas_init`` utility directory: @@ -302,10 +302,10 @@ Run the driver script: ./driver..sh -Check that results are properly generated in ``$EXTRACT_DIR`` and ``$OUTDIR``. +Check that results are properly generated in ``$EXTRACT_DIR`` and ``$OUTDIR``. This can take some time, so if there is only an empty ``$EXTRACT_DIR``, run ``squeue -u $USER`` (on systems with Slurm) or ``qstat -u $USER`` (on systems with PBS Pro) to ensure that the job is still running. Eventually, both directories will contain a file named ``gfs.20251015``. -Create Idealized SST Profile ------------------------------ +Create Idealized SST Profile Using Aquaplanet Tools +----------------------------------------------------- Navigate to the SST profile tool directory: From d99c9ba71934e7923161c7189c30a65304b054fe Mon Sep 17 00:00:00 2001 From: Gillian Petro Date: Wed, 18 Feb 2026 13:11:54 -0500 Subject: [PATCH 21/32] add AP_EXPT env var --- doc/UsersGuide/source/AquaPlanet.rst | 56 +++++++++++++++++++--------- 1 file changed, 38 insertions(+), 18 deletions(-) diff --git a/doc/UsersGuide/source/AquaPlanet.rst b/doc/UsersGuide/source/AquaPlanet.rst index 67f1a33a45..2e534a4a2e 100644 --- a/doc/UsersGuide/source/AquaPlanet.rst +++ b/doc/UsersGuide/source/AquaPlanet.rst @@ -186,6 +186,26 @@ At a high level, the steps are: 5. Apply minor source code changes and recompile the model. 6. Run a 90-day spin-up simulation (in three 30-day restart segments). +Create Working Directory (optional) +------------------------------------- + +Users can create a new directory for their aquaplanet experiment or use an existing directory: + +.. code-block:: console + + mkdir /path/to/ap-expt + cd /path/to/ap-expt + +where ``/path/to/ap-expt`` is the path to the directory where the user plans to perform the following steps (e.g., ``/Users/Joe.Schmoe/ap-expt``). + +Optionally, users can save this directory path in an environment variable (e.g., ``${AP_EXPT}``) to avoid typing out full path names later. + +.. code-block:: console + + export AP_EXPT=$PWD + +In this documentation, ``${AP_EXPT}`` is used, but users are welcome to choose another name for this variable if they prefer. + Clone Required Repositories ---------------------------- @@ -201,7 +221,7 @@ Clone UFS_UTILS: git clone --recursive https://github.com/ufs-community/UFS_UTILS.git -Clone the aquaplanet tools: +Clone the Aquaplanet tools: .. code-block:: console @@ -214,7 +234,7 @@ Navigate to the UFS Weather Model test directory: .. code-block:: console - cd ufs-weather-model/tests/ + cd ${AP_EXPT}/ufs-weather-model/tests/ Edit ``rt.conf`` to include only the following two lines: @@ -238,7 +258,7 @@ Compile the UFS_UTILS tools: .. code-block:: console - cd UFS_UTILS/ + cd ${AP_EXPT}/UFS_UTILS/ module purge module use $PWD/modulefiles module load build..intelllvm @@ -311,7 +331,7 @@ Navigate to the SST profile tool directory: .. code-block:: console - cd ~/Aquaplanet/sst-profile/ + cd ${AP_EXPT}/Aquaplanet/sst-profile/ Copy the SST climatology file from your UFS Weather Model run directory: @@ -339,7 +359,7 @@ Navigate to the glacier tool directory: .. code-block:: console - cd ~/Aquaplanet/glacier/ + cd ${AP_EXPT}/Aquaplanet/glacier/ Copy the glacier file from your run directory: @@ -367,7 +387,7 @@ Navigate to the ice monthly tool directory: .. code-block:: console - cd ~/Aquaplanet/ice-monthly/ + cd ${AP_EXPT}/Aquaplanet/ice-monthly/ Copy the ice climatology file from your run directory: @@ -399,7 +419,7 @@ Navigate to the sea-land mask tool directory: .. code-block:: console - cd ~/Aquaplanet/slmask/ + cd ${AP_EXPT}/Aquaplanet/slmask/ Copy the sea-land mask file from your run directory: @@ -427,7 +447,7 @@ Navigate to the atmospheric profile tool directory: .. code-block:: console - cd ~/Aquaplanet/atmos-profile + cd ${AP_EXPT}/Aquaplanet/atmos-profile Copy the atmospheric analysis file from ``$EXTRACT_DIR``: @@ -465,7 +485,7 @@ Navigate to the surface profile tool directory: .. code-block:: console - cd ~/Aquaplanet/sfc-profile/ + cd ${AP_EXPT}/Aquaplanet/sfc-profile/ Copy the surface analysis file from ``$EXTRACT_DIR``: @@ -503,7 +523,7 @@ Navigate to the orography profile tool directory: .. code-block:: console - cd ~/Aquaplanet/orog-profile/ + cd ${AP_EXPT}/Aquaplanet/orog-profile/ Compile the tool: @@ -528,7 +548,7 @@ Navigate back to the ``gdas_init`` utility directory: .. code-block:: console - cd ~/UFS_UTILS/util/gdas_init/ + cd ${AP_EXPT}/UFS_UTILS/util/gdas_init/ .. important:: @@ -565,7 +585,7 @@ Since the GFS_v17 physics suite is used, empty fields (``sheleg``, ``snwdph``, a .. code-block:: console - cd ~/Aquaplanet/noah-MP-vars/ + cd ${AP_EXPT}/Aquaplanet/noah-MP-vars/ Copy the surface files from the output directory: @@ -600,12 +620,12 @@ Copy the modified orography files: .. code-block:: console cd /INPUT/ - cp ~/UFS_UTILS/fix/orog/C48/C48.mx500_oro_data.tile1.nc oro_data.tile1.nc - cp ~/UFS_UTILS/fix/orog/C48/C48.mx500_oro_data.tile2.nc oro_data.tile2.nc - cp ~/UFS_UTILS/fix/orog/C48/C48.mx500_oro_data.tile3.nc oro_data.tile3.nc - cp ~/UFS_UTILS/fix/orog/C48/C48.mx500_oro_data.tile4.nc oro_data.tile4.nc - cp ~/UFS_UTILS/fix/orog/C48/C48.mx500_oro_data.tile5.nc oro_data.tile5.nc - cp ~/UFS_UTILS/fix/orog/C48/C48.mx500_oro_data.tile6.nc oro_data.tile6.nc + cp ${AP_EXPT}/UFS_UTILS/fix/orog/C48/C48.mx500_oro_data.tile1.nc oro_data.tile1.nc + cp ${AP_EXPT}/UFS_UTILS/fix/orog/C48/C48.mx500_oro_data.tile2.nc oro_data.tile2.nc + cp ${AP_EXPT}/UFS_UTILS/fix/orog/C48/C48.mx500_oro_data.tile3.nc oro_data.tile3.nc + cp ${AP_EXPT}/UFS_UTILS/fix/orog/C48/C48.mx500_oro_data.tile4.nc oro_data.tile4.nc + cp ${AP_EXPT}/UFS_UTILS/fix/orog/C48/C48.mx500_oro_data.tile5.nc oro_data.tile5.nc + cp ${AP_EXPT}/UFS_UTILS/fix/orog/C48/C48.mx500_oro_data.tile6.nc oro_data.tile6.nc Modify Model Source Code ------------------------- From 1cbf220f1d3607e8d5ef2381a10e46bae2b7af98 Mon Sep 17 00:00:00 2001 From: Joshua Kublnick Date: Wed, 18 Feb 2026 13:22:53 -0500 Subject: [PATCH 22/32] Renaming .rst file --- doc/UsersGuide/source/{AquaPlanet.rst => Aquaplanet.rst} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename doc/UsersGuide/source/{AquaPlanet.rst => Aquaplanet.rst} (100%) diff --git a/doc/UsersGuide/source/AquaPlanet.rst b/doc/UsersGuide/source/Aquaplanet.rst similarity index 100% rename from doc/UsersGuide/source/AquaPlanet.rst rename to doc/UsersGuide/source/Aquaplanet.rst From 61c54a314dc054eb0c6803b5489dc98dd30c02ec Mon Sep 17 00:00:00 2001 From: Gillian Petro Date: Wed, 18 Feb 2026 13:27:50 -0500 Subject: [PATCH 23/32] update out_pth -> data_path --- doc/UsersGuide/source/Aquaplanet.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/UsersGuide/source/Aquaplanet.rst b/doc/UsersGuide/source/Aquaplanet.rst index b6add78687..4663d17c87 100644 --- a/doc/UsersGuide/source/Aquaplanet.rst +++ b/doc/UsersGuide/source/Aquaplanet.rst @@ -144,11 +144,11 @@ To use the plotting script with user-generated data: cd ${UFS_WM}/tests-dev/run_dir/aquaplanet_intel cp ${UFS_WM}/tests-dev/test_cases/utils/plot_aq.sh . -2. Edit the variable ``out_pth`` to point to your output location: +2. Edit the variable ``data_path`` to point to your output location: .. code-block:: bash - out_pth=path/to/ufs-weather-model/tests-dev/run_dir/aquaplanet_intel + data_path=path/to/ufs-weather-model/tests-dev/run_dir/aquaplanet_intel 3. Adjust the seasonal timing variables (``winter_start``, ``spring_start``, etc.), which are given in hours from the start of the 90-day spin-up run (hour 0). From 088575b062acff265cccc4381789665c7c566ec3 Mon Sep 17 00:00:00 2001 From: Gillian Petro Date: Thu, 19 Feb 2026 00:01:29 -0500 Subject: [PATCH 24/32] misc updates esp for Advanced section --- doc/UsersGuide/source/Aquaplanet.rst | 62 ++++++++++++++++++++-------- 1 file changed, 45 insertions(+), 17 deletions(-) diff --git a/doc/UsersGuide/source/Aquaplanet.rst b/doc/UsersGuide/source/Aquaplanet.rst index 4663d17c87..bcb6047a21 100644 --- a/doc/UsersGuide/source/Aquaplanet.rst +++ b/doc/UsersGuide/source/Aquaplanet.rst @@ -87,6 +87,14 @@ Navigate to the run directory (``${UFS_WM}/tests-dev/run_dir``). For each 3-mont nhours_fcst: 8640 fhrot: 6480 +Then, update the walltime in the ``job_card`` to run for approximately 8 hours (480 minutes): + +.. code-block:: bash + + #SBATCH --time=480 + +Some systems may succeed in less time; others may need more time, but this is a reasonable starting point. + After each segment completes, rename and move restart files to the ``INPUT`` directory: .. code-block:: console @@ -181,7 +189,7 @@ At a high level, the steps are: 1. Build the UFS Weather Model and run a baseline ``control_c48`` test to generate a run directory. 2. Build UFS_UTILS and use ``gdas_init`` to generate raw atmospheric initial conditions. -3. Use the `Aquaplanet tools `_ to modify SST, ice, sea-land mask, orography, and atmospheric/surface profiles to represent an aquaplanet. +3. Use the `Aquaplanet tools `_ to modify SST, ice, sea-land mask, orography, and atmospheric/surface profiles to represent an aquaplanet. 4. Regenerate initial conditions with the modified files. 5. Apply minor source code changes and recompile the model. 6. Run a 90-day spin-up simulation (in three 30-day restart segments). @@ -225,7 +233,7 @@ Clone the Aquaplanet tools: .. code-block:: console - git clone https://github.com/RatkoVasic-NOAA/Aquaplanet + git clone https://github.com/NOAA-EPIC/Aquaplanet Build UFS Weather Model ------------------------ @@ -247,9 +255,9 @@ Execute the regression test to compile the model and create a baseline run direc .. code-block:: console - ./rt.sh -a -k + ./rt.sh -a -k -e -Replace ```` with your project code (e.g., ``epic``). Save the location of the ``control_c48_intel`` run directory for later use. +Replace ```` with your project code (e.g., ``epic``). Save the location of the ``control_c48_intel`` run directory for later use. The ``-e`` flag (or the ``-r``) flag are not required. Build UFS_UTILS --------------- @@ -283,6 +291,8 @@ Since orography files need to be edited, remove the link and copy the files. The cp /scratch3/NCEPDEV/global/role.glopara/fix/orog/20240917/*.dat orog/. cp -r /scratch3/NCEPDEV/global/role.glopara/fix/orog/20240917/C48/ orog/. +.. _gen-initial-atmos-data: + Generate Initial Atmospheric Data With UFS_UTILS -------------------------------------------------- @@ -449,11 +459,11 @@ Navigate to the atmospheric profile tool directory: cd ${AP_EXPT}/Aquaplanet/atmos-profile -Copy the atmospheric analysis file from ``$EXTRACT_DIR``: +Copy the atmospheric analysis file from ``$EXTRACT_DIR`` (set in :numref:`Step %s ` to ``${AP_EXPT}/UFS_UTILS/input``): .. code-block:: console - cp $EXTRACT_DIR/gfs.t06z.atmanl.nc . + cp ${EXTRACT_DIR}/gfs.20251015/06/atmos/gfs.t06z.atmanl.nc . Compile and load required modules: @@ -487,11 +497,11 @@ Navigate to the surface profile tool directory: cd ${AP_EXPT}/Aquaplanet/sfc-profile/ -Copy the surface analysis file from ``$EXTRACT_DIR``: +Copy the surface analysis file from ``$EXTRACT_DIR`` (set in :numref:`Step %s ` to ``${AP_EXPT}/UFS_UTILS/input``): .. code-block:: console - cp $EXTRACT_DIR/gfs.t06z.sfcanl.nc . + cp ${EXTRACT_DIR}/gfs.20251015/06/atmos/gfs.t06z.sfcanl.nc . Compile and load required modules: @@ -552,7 +562,7 @@ Navigate back to the ``gdas_init`` utility directory: .. important:: - In the ``config`` file, make sure that ``EXTRACT_DATA`` is set to ``no`` since data is already extracted. + In the ``config`` file, set ``EXTRACT_DATA`` to ``no`` since data is already extracted. Delete or rename the output directory. For example: @@ -572,7 +582,7 @@ Run the driver script: ./driver..sh -Check the results in: +Once the job has finished running, check the results in: .. code-block:: console @@ -587,17 +597,27 @@ Since the GFS_v17 physics suite is used, empty fields (``sheleg``, ``snwdph``, a cd ${AP_EXPT}/Aquaplanet/noah-MP-vars/ -Copy the surface files from the output directory: +Copy the surface files from the output directory (set in :numref:`Step %s ` to ``${AP_EXPT}/UFS_UTILS/output``): .. code-block:: console cp $OUTDIR/gfs.20251015/06/model/atmos/input/sfc_* . -Compile and run: +Compile and load required modules: .. code-block:: console ./compile.sh + module purge + module use /contrib/spack-stack/spack-stack-1.9.2/envs/ue-oneapi-2024.2.1/install/modulefiles/Core + module load stack-oneapi/2024.2.1 + module load stack-intel-oneapi-mpi/2021.13 + module load netcdf-fortran/4.6.1 + +Run the tool: + +.. code-block:: console + ./add-vars.x Copy the modified files back: @@ -609,7 +629,7 @@ Copy the modified files back: Copy Initial Conditions to Run Directory ----------------------------------------- -Copy all generated initial condition files to your run directory: +Copy all generated initial condition files from ``${OUTDIR}`` (set in to ``${AP_EXPT}/UFS_UTILS/output`` in :numref:`Step %s `) to your run directory: .. code-block:: console @@ -645,7 +665,7 @@ Edit the file ``./ufs-weather-model/UFSATM/ccpp/physics/physics/Radiation/radiat **Reduce Land-Surface Warnings** -Edit the file ``./ufs-weather-model/UFSATM/fv3/atmos_cubed_sphere/tools/fv_diagnostics.F90``. After lines 4292 and 4349, add: +Edit the file ``./ufs-weather-model/UFSATM/fv3/atmos_cubed_sphere/tools/fv_diagnostics.F90``. After lines 4292 and 4349, where it says ``if ( present(bad_range) ) then``, add: .. code-block:: console @@ -653,7 +673,17 @@ Edit the file ``./ufs-weather-model/UFSATM/fv3/atmos_cubed_sphere/tools/fv_diagn return ! AQUAPLANET TEST CASE -Recompile the model after making these changes and update the executable in your run directory. +Recompile the model after making these changes. To do this, users can comment out the ``RUN`` line in ``rt.conf`` and run the ``rt.sh`` script: + +.. code-block:: console + + ./rt.sh -a -k -e + +Then, update the executable in your run directory by copying the newly compiled executable in your test directory (``fv3_atm_dyn32_intel.exe``) to your run directory, where it is named ``fv3.exe``. + +.. code-block:: console + + cp ${AP_EXPT}/ufs-weather-model/tests/fv3_atm_dyn32_intel.exe /fv3.exe Running the 90-Day Spin-Up --------------------------- @@ -747,5 +777,3 @@ Submit the job: After this final segment completes, the files in the ``RESTART`` directory represent spun-up initial conditions. Rename and copy these files to the ``INPUT`` directory --- these are the new initial conditions for your experiment (spun up for 90 days). Save them for future use. To continue running experiments with these self-generated initial conditions, follow the instructions in :numref:`Section %s `, using the ``input.nml`` from the restart runs and updating ``fhrot`` and ``nhours_fcst`` in ``model_configure`` as described in the extended experiments section. - -.. include:: ./doc-snippets/hsd_notes.rst \ No newline at end of file From 7de319515f4874b8be9ffb0f515cfbd58bbd97ef Mon Sep 17 00:00:00 2001 From: Gillian Petro Date: Thu, 19 Feb 2026 14:19:49 -0500 Subject: [PATCH 25/32] minor updates, explicit build commands, use RUNDIR --- doc/UsersGuide/source/Aquaplanet.rst | 47 ++++++++++++++++++---------- 1 file changed, 30 insertions(+), 17 deletions(-) diff --git a/doc/UsersGuide/source/Aquaplanet.rst b/doc/UsersGuide/source/Aquaplanet.rst index bcb6047a21..5c962e1138 100644 --- a/doc/UsersGuide/source/Aquaplanet.rst +++ b/doc/UsersGuide/source/Aquaplanet.rst @@ -138,7 +138,7 @@ By default, this script creates seasonal means for three variables: - Precipitation patterns - Temperature at 500 mb -The script uses staged data from a 1-year control simulation. If you want to use this dataset on other machines (which have access to :term:`HPSS`), you can retrieve it from HPSS at ``/5year/NCEPDEV/emc-meso/Ratko.Vasic/AQUAPLANET/1yr-results.tar``. +The script uses staged data from a 1-year control simulation. If you want to use this dataset on other machines (which have access to :term:`HPSS`), you can retrieve it from HPSS at ``/5year/NCEPDEV/emc-meso/Ratko.Vasic/AQUAPLANET/1yr-results.tar``. Alternatively, users can adjust the ``data_path`` variable to point to their own model output from an extended run. Customizing the Plotting Script -------------------------------- @@ -152,7 +152,7 @@ To use the plotting script with user-generated data: cd ${UFS_WM}/tests-dev/run_dir/aquaplanet_intel cp ${UFS_WM}/tests-dev/test_cases/utils/plot_aq.sh . -2. Edit the variable ``data_path`` to point to your output location: +2. Edit the variable ``data_path`` to point to either the prestaged model output data on Ursa or the data generated from your own model run. For example: .. code-block:: bash @@ -343,11 +343,11 @@ Navigate to the SST profile tool directory: cd ${AP_EXPT}/Aquaplanet/sst-profile/ -Copy the SST climatology file from your UFS Weather Model run directory: +Copy the SST climatology file from your UFS Weather Model run directory, which will be referred to as ``${RUNDIR}`` in this documentation: .. code-block:: console - cp /RTGSST.1982.2012.monthly.clim.grb . + cp ${RUNDIR}/RTGSST.1982.2012.monthly.clim.grb . Compile and run the tool: @@ -360,7 +360,7 @@ Copy the modified file back to your run directory: .. code-block:: console - cp new-RTGSST.1982.2012.monthly.clim.grb /RTGSST.1982.2012.monthly.clim.grb + cp new-RTGSST.1982.2012.monthly.clim.grb ${RUNDIR}/RTGSST.1982.2012.monthly.clim.grb Edit Global Ice Data -------------------- @@ -375,7 +375,7 @@ Copy the glacier file from your run directory: .. code-block:: console - cp /global_glacier.2x2.grb . + cp ${RUNDIR}/global_glacier.2x2.grb . Compile and run the tool: @@ -388,7 +388,7 @@ Copy the modified file back: .. code-block:: console - cp new-global_glacier.2x2.grb /global_glacier.2x2.grb + cp new-global_glacier.2x2.grb ${RUNDIR}/global_glacier.2x2.grb Edit Monthly Ice Data --------------------- @@ -403,7 +403,7 @@ Copy the ice climatology file from your run directory: .. code-block:: console - cp /IMS-NIC.blended.ice.monthly.clim.grb . + cp ${RUNDIR}/IMS-NIC.blended.ice.monthly.clim.grb . Compile and run the tool: @@ -420,7 +420,7 @@ Copy the modified file back: .. code-block:: console - cp new-IMS-NIC.blended.ice.monthly.clim.grb /IMS-NIC.blended.ice.monthly.clim.grb + cp new-IMS-NIC.blended.ice.monthly.clim.grb ${RUNDIR}/IMS-NIC.blended.ice.monthly.clim.grb Edit Sea-Land Mask Data ----------------------- @@ -435,7 +435,7 @@ Copy the sea-land mask file from your run directory: .. code-block:: console - cp /global_slmask.t62.192.94.grb . + cp ${RUNDIR}/global_slmask.t62.192.94.grb . Compile and run the tool: @@ -448,7 +448,7 @@ Copy the modified file back: .. code-block:: console - cp new-global_slmask.t62.192.94.grb /global_slmask.t62.192.94.grb + cp new-global_slmask.t62.192.94.grb ${RUNDIR}/global_slmask.t62.192.94.grb Setup Atmospheric Profiles --------------------------- @@ -633,13 +633,13 @@ Copy all generated initial condition files from ``${OUTDIR}`` (set in to ``${AP_ .. code-block:: console - cp $OUTDIR/gfs.20251015/06/model/atmos/input/* /INPUT/. + cp $OUTDIR/gfs.20251015/06/model/atmos/input/* ${RUNDIR}/INPUT/. Copy the modified orography files: .. code-block:: console - cd /INPUT/ + cd ${RUNDIR}/INPUT/ cp ${AP_EXPT}/UFS_UTILS/fix/orog/C48/C48.mx500_oro_data.tile1.nc oro_data.tile1.nc cp ${AP_EXPT}/UFS_UTILS/fix/orog/C48/C48.mx500_oro_data.tile2.nc oro_data.tile2.nc cp ${AP_EXPT}/UFS_UTILS/fix/orog/C48/C48.mx500_oro_data.tile3.nc oro_data.tile3.nc @@ -673,17 +673,24 @@ Edit the file ``./ufs-weather-model/UFSATM/fv3/atmos_cubed_sphere/tools/fv_diagn return ! AQUAPLANET TEST CASE -Recompile the model after making these changes. To do this, users can comment out the ``RUN`` line in ``rt.conf`` and run the ``rt.sh`` script: +Recompile the model after making these changes. To do this, run the following commands: .. code-block:: console - ./rt.sh -a -k -e + module use ${AP_EXPT}/ufs-weather-model/modulefiles + module load ufs__intel + cd ${AP_EXPT}/ufs-weather-model + mkdir build + cd build + cmake .. -DAPP=ATM -D32BIT=ON -DCCPP_SUITES=FV3_GFS_v17_p8_ugwpv1 + make -j4 -Then, update the executable in your run directory by copying the newly compiled executable in your test directory (``fv3_atm_dyn32_intel.exe``) to your run directory, where it is named ``fv3.exe``. +This will produce an executable called ``ufs_model``. Rename this executable to ``fv3.exe``. Then, update the executable in your run directory by copying the newly compiled executable in your build directory to your run directory. .. code-block:: console - cp ${AP_EXPT}/ufs-weather-model/tests/fv3_atm_dyn32_intel.exe /fv3.exe + cp ufs_model fv3.exe + mv fv3.exe ${RUNDIR} Running the 90-Day Spin-Up --------------------------- @@ -715,6 +722,12 @@ Submit the job: sbatch job_card +In general, each 30-day run will take 5.5-6 hours, but this can vary. User can view progress with the ``tail`` command: + +.. code-block:: console + + tail -f out + **First Restart (Days 31-60)** After the first month completes, prepare for the restart: From 09832b024808a24a03ab14f2e176c7aadaebb35a Mon Sep 17 00:00:00 2001 From: Gillian Petro Date: Thu, 19 Feb 2026 14:28:08 -0500 Subject: [PATCH 26/32] make cmd update --- doc/UsersGuide/source/Aquaplanet.rst | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/doc/UsersGuide/source/Aquaplanet.rst b/doc/UsersGuide/source/Aquaplanet.rst index 5c962e1138..6b6ca531d1 100644 --- a/doc/UsersGuide/source/Aquaplanet.rst +++ b/doc/UsersGuide/source/Aquaplanet.rst @@ -685,7 +685,9 @@ Recompile the model after making these changes. To do this, run the following co cmake .. -DAPP=ATM -D32BIT=ON -DCCPP_SUITES=FV3_GFS_v17_p8_ugwpv1 make -j4 -This will produce an executable called ``ufs_model``. Rename this executable to ``fv3.exe``. Then, update the executable in your run directory by copying the newly compiled executable in your build directory to your run directory. +Note that users can leave off the ``-j`` flag after the ``make`` command or change the integer number after it to run with fewer or more processors. + +Running these commands will produce an executable called ``ufs_model``. Rename this executable to ``fv3.exe``. Then, update the executable in your run directory by copying the newly compiled executable in your build directory to your run directory. .. code-block:: console From 906554a2e339dd2b907f98279f48288f31ce77b3 Mon Sep 17 00:00:00 2001 From: Gillian Petro Date: Thu, 19 Feb 2026 16:09:40 -0500 Subject: [PATCH 27/32] misc updates --- doc/UsersGuide/source/Aquaplanet.rst | 69 +++++++++++++++++++++------- 1 file changed, 53 insertions(+), 16 deletions(-) diff --git a/doc/UsersGuide/source/Aquaplanet.rst b/doc/UsersGuide/source/Aquaplanet.rst index 6b6ca531d1..3a66b2d130 100644 --- a/doc/UsersGuide/source/Aquaplanet.rst +++ b/doc/UsersGuide/source/Aquaplanet.rst @@ -19,13 +19,15 @@ Obtaining Data for HSD Cases .. include:: ./doc-snippets/hsd_data.rst +Note that the 90-day spin-up for the aquaplanet case has already been completed for this data. Advanced users may prefer to try the 90-day spin-up themselves using the instructions in :numref:`Section %s `. However, it is recommended that all users try the case first with the prestaged data. + .. _run-aquaplanet: ============================== Running the Aquaplanet Case ============================== -This section explains how to run the Aquaplanet case using the ``ufs_test.sh`` script with pre-staged initial conditions. This is the recommended way to run the case for most users. +This section explains how to run the Aquaplanet case using the ``ufs_test.sh`` script with prestaged initial conditions. This is the recommended way to run the case for most users. Clone the Repository -------------------- @@ -64,7 +66,19 @@ Running Extended Experiments After running the base test case, users can extend the simulation by running multiple restart segments. For example, to run a 1-year experiment broken into four 3-month segments: -Navigate to the run directory (``${UFS_WM}/tests-dev/run_dir``). For each 3-month segment, update ``fhrot`` and ``nhours_fcst`` in ``model_configure``: +Navigate to the run directory: + +.. code-block:: console + + cd ${UFS_WM}/tests-dev/run_dir/aquaplanet_intel + +Users may optionally save this location in an environment variable(e.g., ``${RUNDIR}``): + +.. code-block:: console + + export RUNDIR=${UFS_WM}/tests-dev/run_dir/aquaplanet_intel + +For each 3-month segment, update ``fhrot`` and ``nhours_fcst`` in ``model_configure``: **First segment (months 4-6):** @@ -107,14 +121,33 @@ After each segment completes, rename and move restart files to the ``INPUT`` dir Checking Results ---------------- -.. include:: ./doc-snippets/hsd_check_results.rst - -For example, to monitor progress or check results for the ``aquaplanet`` case, run: +To monitor progress or check results for the ``aquaplanet`` case, run: .. code-block:: console - tail -f ${UFS_WM}/tests-dev/run_dir/aquaplanet_intel/err tail -f ${UFS_WM}/tests-dev/run_dir/aquaplanet_intel/out + tail -f ${UFS_WM}/tests-dev/run_dir/aquaplanet_intel/err + +The ``out`` file contains messages output during the model run. The ``err`` file reports on errors and is useful if the case does not finish properly. Additionally, the model will produce ``atmf*.nc`` and ``sfcf*.nc`` files every 24 hours. + +When the test case finishes running, the ``out`` file should include a resource statistics summary at the bottom: + +.. code-block:: console + + 0: *****************RESOURCE STATISTICS******************************* + 0: The total amount of wall time = 22639.581802 + 0: The total amount of time in user mode = 22222.145691 + 0: The total amount of time in sys mode = 189.322066 + 0: The maximum resident set size (KB) = 1769600 + 0: Number of page faults without I/O activity = 89229617 + 0: Number of page faults with I/O activity = 99093 + 0: Number of times filesystem performed INPUT = 1368912 + 0: Number of times filesystem performed OUTPUT = 115520 + 0: Number of Voluntary Context Switches = 347098 + 0: Number of InVoluntary Context Switches = 758822 + 0: *****************END OF RESOURCE STATISTICS************************* + 0: + Model ended: Thu Feb 19 10:46:07 UTC 2026 .. _plotting-aquaplanet: @@ -156,7 +189,9 @@ To use the plotting script with user-generated data: .. code-block:: bash - data_path=path/to/ufs-weather-model/tests-dev/run_dir/aquaplanet_intel + data_path=./ + + This will use data from the run directory (rather than data from the prestaged model output) to produce the plots. 3. Adjust the seasonal timing variables (``winter_start``, ``spring_start``, etc.), which are given in hours from the start of the 90-day spin-up run (hour 0). @@ -181,14 +216,14 @@ Advanced: Setting Up the Aquaplanet Experiment from Scratch .. note:: - This section is **optional**. Most users can run the Aquaplanet case using the ``ufs_test.sh`` method described above, which uses pre-staged initial conditions. The steps below are for advanced users who wish to create their own initial conditions from scratch. + This section is **optional**. Most users can run the Aquaplanet case using the ``ufs_test.sh`` method described above, which uses pre-staged initial conditions. The steps below are for advanced users who wish to create their own initial conditions from scratch. :term:`HPSS` access is required for the UFS_UTILS steps. The from-scratch setup involves editing orography, SST, ice, and sea-land mask files to create an aquaplanet configuration, then running a 90-day spin-up to allow the model to reach a balanced state. This process produces the same initial conditions that are provided pre-staged for the standard test case. At a high level, the steps are: 1. Build the UFS Weather Model and run a baseline ``control_c48`` test to generate a run directory. -2. Build UFS_UTILS and use ``gdas_init`` to generate raw atmospheric initial conditions. +2. Build UFS_UTILS, and use ``gdas_init`` to generate raw atmospheric initial conditions. 3. Use the `Aquaplanet tools `_ to modify SST, ice, sea-land mask, orography, and atmospheric/surface profiles to represent an aquaplanet. 4. Regenerate initial conditions with the modified files. 5. Apply minor source code changes and recompile the model. @@ -197,7 +232,7 @@ At a high level, the steps are: Create Working Directory (optional) ------------------------------------- -Users can create a new directory for their aquaplanet experiment or use an existing directory: +Users can create a new directory for their aquaplanet experiment or use an existing directory. To create a new directory, run: .. code-block:: console @@ -257,7 +292,7 @@ Execute the regression test to compile the model and create a baseline run direc ./rt.sh -a -k -e -Replace ```` with your project code (e.g., ``epic``). Save the location of the ``control_c48_intel`` run directory for later use. The ``-e`` flag (or the ``-r``) flag are not required. +Replace ```` with your project code (e.g., ``epic``). Save the location of the ``control_c48_intel`` run directory for later use. The ``-e`` flag (or the ``-r``) flag are not required but can be useful. Build UFS_UTILS --------------- @@ -314,6 +349,8 @@ Edit the ``config`` file to set the following variables: CRES_HIRES=C48 OUTDIR=/output/ +where ```` is replaced with the actual path to the UFS_UTILS repository. + Edit the driver script (e.g., ``driver.ursa.sh``) to set: .. code-block:: console @@ -332,7 +369,7 @@ Run the driver script: ./driver..sh -Check that results are properly generated in ``$EXTRACT_DIR`` and ``$OUTDIR``. This can take some time, so if there is only an empty ``$EXTRACT_DIR``, run ``squeue -u $USER`` (on systems with Slurm) or ``qstat -u $USER`` (on systems with PBS Pro) to ensure that the job is still running. Eventually, both directories will contain a file named ``gfs.20251015``. +Check that results are properly generated in ``${EXTRACT_DIR}`` and ``${OUTDIR}``. This can take some time, so if there is only an empty ``${EXTRACT_DIR}``, run ``squeue -u $USER`` (on systems with Slurm) or ``qstat -u $USER`` (on systems with PBS Pro) to ensure that the job is still running. Eventually, both directories will contain a file named ``gfs.20251015``. Create Idealized SST Profile Using Aquaplanet Tools ----------------------------------------------------- @@ -459,7 +496,7 @@ Navigate to the atmospheric profile tool directory: cd ${AP_EXPT}/Aquaplanet/atmos-profile -Copy the atmospheric analysis file from ``$EXTRACT_DIR`` (set in :numref:`Step %s ` to ``${AP_EXPT}/UFS_UTILS/input``): +Copy the atmospheric analysis file from ``${EXTRACT_DIR}`` (set in :numref:`Step %s ` to ``${AP_EXPT}/UFS_UTILS/input``): .. code-block:: console @@ -486,7 +523,7 @@ Copy the modified atmospheric file back to the extract directory: .. code-block:: console - cp gfs.t06z.atmanl.nc $EXTRACT_DIR/gfs.20251015/06/atmos/gfs.t06z.atmanl.nc + cp gfs.t06z.atmanl.nc ${EXTRACT_DIR}/gfs.20251015/06/atmos/gfs.t06z.atmanl.nc Setup Surface Profiles ----------------------- @@ -497,7 +534,7 @@ Navigate to the surface profile tool directory: cd ${AP_EXPT}/Aquaplanet/sfc-profile/ -Copy the surface analysis file from ``$EXTRACT_DIR`` (set in :numref:`Step %s ` to ``${AP_EXPT}/UFS_UTILS/input``): +Copy the surface analysis file from ``${EXTRACT_DIR}`` (set in :numref:`Step %s ` to ``${AP_EXPT}/UFS_UTILS/input``): .. code-block:: console @@ -524,7 +561,7 @@ Copy the modified surface file back to the extract directory: .. code-block:: console - cp gfs.t06z.sfcanl.nc $EXTRACT_DIR/gfs.20251015/06/atmos/gfs.t06z.sfcanl.nc + cp gfs.t06z.sfcanl.nc ${EXTRACT_DIR}/gfs.20251015/06/atmos/gfs.t06z.sfcanl.nc Prepare Orography Fields ------------------------- From 11b9394390ebc4236cfa1327cbf81b47a0308528 Mon Sep 17 00:00:00 2001 From: Joshua Kublnick <47824899+jdkublnick@users.noreply.github.com> Date: Fri, 20 Feb 2026 10:41:50 -0500 Subject: [PATCH 28/32] Update doc/UsersGuide/source/Aquaplanet.rst Co-authored-by: Gillian Petro <96886803+gspetro-NOAA@users.noreply.github.com> --- doc/UsersGuide/source/Aquaplanet.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/doc/UsersGuide/source/Aquaplanet.rst b/doc/UsersGuide/source/Aquaplanet.rst index 3a66b2d130..8c6699eec6 100644 --- a/doc/UsersGuide/source/Aquaplanet.rst +++ b/doc/UsersGuide/source/Aquaplanet.rst @@ -777,6 +777,7 @@ After the first month completes, prepare for the restart: for file in 20*.060000.*.nc; do mv "$file" "${file#20*.060000.}"; done mv 20*.060000.coupler.res coupler.res mv * ../INPUT/. + cd .. Edit ``input.nml`` and set: From 076e593bb774d4f8dce21b5767f095280da459c7 Mon Sep 17 00:00:00 2001 From: Gillian Petro <96886803+gspetro-NOAA@users.noreply.github.com> Date: Fri, 20 Feb 2026 11:59:52 -0500 Subject: [PATCH 29/32] minor add --- doc/UsersGuide/source/Aquaplanet.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/doc/UsersGuide/source/Aquaplanet.rst b/doc/UsersGuide/source/Aquaplanet.rst index 8c6699eec6..cbdb700873 100644 --- a/doc/UsersGuide/source/Aquaplanet.rst +++ b/doc/UsersGuide/source/Aquaplanet.rst @@ -813,6 +813,7 @@ After the second month completes: for file in 20*.060000.*.nc; do mv "$file" "${file#20*.060000.}"; done mv 20*.060000.coupler.res coupler.res mv * ../INPUT/. + cd .. Keep ``input.nml`` settings from the first restart. Edit ``model_configure`` and set: From bc2415823f2caca27aee797f56f5b8538c9d9a8e Mon Sep 17 00:00:00 2001 From: Gillian Petro <96886803+gspetro-NOAA@users.noreply.github.com> Date: Fri, 20 Feb 2026 12:49:25 -0500 Subject: [PATCH 30/32] fix docker img path --- doc/UsersGuide/source/HSDcontainer.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/UsersGuide/source/HSDcontainer.rst b/doc/UsersGuide/source/HSDcontainer.rst index c474dbab01..1c8f4b479d 100644 --- a/doc/UsersGuide/source/HSDcontainer.rst +++ b/doc/UsersGuide/source/HSDcontainer.rst @@ -148,7 +148,7 @@ To build the container from a Docker image, users can run: .. code-block:: console - singularity build --force ubuntu22.04-intel-ue-1.9.2-wm-hsd.img docker://noaaepic/ubuntu22.04-intel-ue-1.9.2-wm-hsd.img:ue192-hsd-aquaplanet + singularity build --force ubuntu22.04-intel-ue-1.9.2-wm-hsd.img docker://noaaepic/ubuntu22.04-intel2024.2.0-1-devel-wm:ue192-hsd-aquaplanet This process may take several hours depending on the system. From e3489f6225f9192fdd96d0ae377bf0c2ea7483ed Mon Sep 17 00:00:00 2001 From: Gillian Petro <96886803+gspetro-NOAA@users.noreply.github.com> Date: Fri, 20 Feb 2026 12:56:01 -0500 Subject: [PATCH 31/32] Extended run clarification Added job submission instructions and file handling steps. --- doc/UsersGuide/source/Aquaplanet.rst | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/doc/UsersGuide/source/Aquaplanet.rst b/doc/UsersGuide/source/Aquaplanet.rst index cbdb700873..ad81a6ee5c 100644 --- a/doc/UsersGuide/source/Aquaplanet.rst +++ b/doc/UsersGuide/source/Aquaplanet.rst @@ -107,6 +107,12 @@ Then, update the walltime in the ``job_card`` to run for approximately 8 hours ( #SBATCH --time=480 +Submit the job: + +.. code-block:: console + + sbatch job_card + Some systems may succeed in less time; others may need more time, but this is a reasonable starting point. After each segment completes, rename and move restart files to the ``INPUT`` directory: @@ -117,6 +123,9 @@ After each segment completes, rename and move restart files to the ``INPUT`` dir for file in 20*.060000.*.nc; do mv "$file" "${file#20*.060000.}"; done mv 20*.060000.coupler.res coupler.res mv * ../INPUT/. + cd .. + +Then, repeat the steps above to adjust ``nhours_fcst`` and ``fhrot`` and submit the job card. Checking Results ---------------- From d046cdad9723916301456554950dbfe6c86da02f Mon Sep 17 00:00:00 2001 From: Gillian Petro <96886803+gspetro-NOAA@users.noreply.github.com> Date: Fri, 20 Feb 2026 13:55:23 -0500 Subject: [PATCH 32/32] reorder two doc lines --- doc/UsersGuide/source/Aquaplanet.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/doc/UsersGuide/source/Aquaplanet.rst b/doc/UsersGuide/source/Aquaplanet.rst index ad81a6ee5c..925916409c 100644 --- a/doc/UsersGuide/source/Aquaplanet.rst +++ b/doc/UsersGuide/source/Aquaplanet.rst @@ -107,14 +107,14 @@ Then, update the walltime in the ``job_card`` to run for approximately 8 hours ( #SBATCH --time=480 -Submit the job: +Some systems may succeed in less time; others may need more time, but this is a reasonable starting point. + +Next, submit the job: .. code-block:: console sbatch job_card -Some systems may succeed in less time; others may need more time, but this is a reasonable starting point. - After each segment completes, rename and move restart files to the ``INPUT`` directory: .. code-block:: console