From 2caf5e101e0777689ab527d3fcd660fcc077550e Mon Sep 17 00:00:00 2001 From: Dom Heinzeller Date: Thu, 5 Mar 2020 08:58:27 -0700 Subject: [PATCH] Remove old documentation from doc/ --- doc/FV3_License_Public_Release_FY18Q2 | 12 -- doc/README.md | 56 ------ doc/README_CENTOS.txt | 235 ------------------------ doc/README_CHEYENNE.txt | 67 ------- doc/README_FV3GFS_Public_Release_FY18Q2 | 5 - doc/README_HERA.txt | 80 -------- doc/README_MACOSX_clanggfortran.txt | 229 ----------------------- doc/README_MACOSX_gccgfortran.txt | 221 ---------------------- doc/README_THEIA.txt | 80 -------- doc/README_UBUNTU.txt | 135 -------------- 10 files changed, 1120 deletions(-) delete mode 100644 doc/FV3_License_Public_Release_FY18Q2 delete mode 100644 doc/README.md delete mode 100644 doc/README_CENTOS.txt delete mode 100644 doc/README_CHEYENNE.txt delete mode 100644 doc/README_FV3GFS_Public_Release_FY18Q2 delete mode 100644 doc/README_HERA.txt delete mode 100644 doc/README_MACOSX_clanggfortran.txt delete mode 100644 doc/README_MACOSX_gccgfortran.txt delete mode 100644 doc/README_THEIA.txt delete mode 100644 doc/README_UBUNTU.txt diff --git a/doc/FV3_License_Public_Release_FY18Q2 b/doc/FV3_License_Public_Release_FY18Q2 deleted file mode 100644 index becd221591..0000000000 --- a/doc/FV3_License_Public_Release_FY18Q2 +++ /dev/null @@ -1,12 +0,0 @@ -Copyright 2018 Princeton University - -The Finite Volume Cubed Sphere (FV3) Dynamic Core Model code was jointly developed by the National Oceanic and Atmospheric Administration (NOAA) and Princeton University. The gold standard copy of the FV3 Model Code will be maintained by NOAA at https://vlab.ncep.noaa.gov/redmine/projects/comfv3/repository - -In cooperation with the Copyright Holder, the NOAA is releasing this code under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. - -You may obtain a copy of the License at - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. -Software code created by U.S. Government employees is not subject to copyright in the United States (17 U.S.C. §105). The United States/Department of Commerce reserve all rights to seek and obtain copyright protection in countries other than the United States for Software authored in its entirety by the Department of Commerce. To this end, the Department of Commerce hereby grants to Recipient a royalty-free, nonexclusive license to use, copy, and create derivative works of the Software outside of the United States. - diff --git a/doc/README.md b/doc/README.md deleted file mode 100644 index 70c6fee81d..0000000000 --- a/doc/README.md +++ /dev/null @@ -1,56 +0,0 @@ -NEMSFV3GFS Build and Test Instructions -====================================== - -This page documents the build system and test suite for the NEMSFV3 -app. This is a NEMS app that runs the standalone (uncoupled) Finite -Volume Model (FV3). - -Supported Builds and Platforms ------------------------------------------------------ - -The NEMSFV3 app currently works on NOAA WCOSS Cray and theia. -No other platforms are tested or supported at this time. -There are plans to port to the WCOSS phase1 and phase2, theia machine. - -The app can be built in the following ways via the the script -compile.sh under tests directory. The simple regression test -suite is set up for senity check when the changes are coming in. - -NEMS top level driver and fv3_cap ------------------------------------------------------------- -NEMSFV3GFS is using NEMS as top level driver, the fv3_cap connects NEMS and -main fv3 code. The basic calling sequence is listed below: - -1)The main program for NEMSfv3gfs is: trunk/NEMS/src/MAIN_NEMS.F90, -which sets up nems grid component, reads date/time in configure file and -sets up central clock for nems grid component. - -2)The initialiation, run and finalzation of NEMS grid component are -in trunk/NEMS/src/module_NEMS_GRID_COMP.F90, it calls earth component. - -3)earth component (trunk/NEMS/src/module_EARTH_GRID_COMP.F90) has -generic setting for the subgrid component defined by ifdef - -#ifdef FRONT_FV3 - use FRONT_FV3, only: FV3_SS => SetServices -#endif - -, and read in nems.configure to decide the run sequence. -For FV3, the SetServices is defined in fv3_cap.F90. - -4)fv3_cap connects nems with fv3. The SetService defines fv3 initialization, -run and finalize phases. -4a)In fv3_cap, the initialization also has several phases, -which is required for coupling with other component. -The fv3 model set up is in one of the initialization phases called InitializeAdvertise, -In this subroutine, fv3 subroutines are called to initialize fms, mpp, and diag_manager, etc. -4b)fv3 run phase is in subroutine ModelAdvance. The forecast integration loop is in ModelAdvance, -fv3 subroutines are called to run fv3 dyn core (update_atmos_model_dynamics), physics -(update_atmos_radiation_physics) etc. -4c)fv3 finalization phase is in subroutine atmos_model_finalize. This subroutine -writes the time stamp for restart and finalizes the mpp, diag_manager and fms. - -GFDL is preparing documentation for fv3 code. For documentation besides NEMS and fv3 cap, -please contact GFDL. - - diff --git a/doc/README_CENTOS.txt b/doc/README_CENTOS.txt deleted file mode 100644 index d819ac4c79..0000000000 --- a/doc/README_CENTOS.txt +++ /dev/null @@ -1,235 +0,0 @@ -# Dom Heinzeller (dom.heinzeller@noaa.gov), 08/21/2019 - -In order to build and run the FV3 trunk (August 2019) with possible CCPP extensions by GMTB on CentOS Linux, -the following installation steps are recommended. The version numbers correspond to the default versions in -August 2019 and will change to newer versions in the future. Unless problems occur during the manual builds in -step 4, these differences can be ignored. It is also assumed that the bash shell is used in the following. - -1. Install CentOS 7 (minimal install or more) or start up cloud instance (e.g. Amazon AWS) with Centos 7 - -2. Install standard packages as root (su) - - yum update - yum install -y net-tools - yum install -y gcc-gfortran - yum install -y gcc-c++ - yum install -y screen - yum install -y wget - yum install -y cmake - yum install -y bzip2 - yum install -y texinfo - yum install -y autogen - yum install -y dejagnu - yum install -y byacc - yum install -y curl-devel - yum install -y m4 - yum install -y git - yum install -y libxml2-devel - -3. Install thirdparty libraries - - mkdir -p /usr/local/src && cd /usr/local/src - - # gcc-8.3.0 - wget http://ftp.mirrorservice.org/sites/sourceware.org/pub/gcc/releases/gcc-8.3.0/gcc-8.3.0.tar.gz - tar -xvf gcc-8.3.0.tar.gz - cd gcc-8.3.0 - ./contrib/download_prerequisites # if this hangs, need to change the ftp URLs to https URLs in the script (behind firewall?) - ./configure \ - --disable-multilib \ - --enable-languages=c,c++,fortran \ - --prefix=/usr/local 2>&1 | tee log.config - make 2>&1 | tee log.make - make install 2>&1 | tee log.install - cd .. - rm -fr gcc-8.3.0 - - export LD_LIBRARY_PATH="/usr/local/lib:/usr/local/lib64:$LD_LIBRARY_PATH" - - # mpich-3.3.1 - wget http://www.mpich.org/static/downloads/3.3.1/mpich-3.3.1.tar.gz - tar -xvzf mpich-3.3.1.tar.gz - cd mpich-3.3.1 - ./configure \ - --prefix=/usr/local 2>&1 | tee log.config - make 2>&1 | tee log.make - make install 2>&1 | tee log.install - cd .. - rm -fr mpich-3.3.1 - - # netcdf-4.7.0 - wget https://www.zlib.net/zlib-1.2.11.tar.gz - wget https://support.hdfgroup.org/ftp/lib-external/szip/2.1.1/src/szip-2.1.1.tar.gz - # go to https://www.hdfgroup.org/downloads/hdf5/source-code/ and download hdf5-1.10.5.tar.bz2 - wget https://www.unidata.ucar.edu/downloads/netcdf/ftp/netcdf-c-4.7.0.tar.gz - wget https://www.unidata.ucar.edu/downloads/netcdf/ftp/netcdf-fortran-4.4.5.tar.gz - # - tar -xvzf zlib-1.2.11.tar.gz - cd zlib-1.2.11 - ./configure \ - --prefix=/usr/local \ - --static 2>&1 | tee log.config - make 2>&1 | tee log.make - make install 2>&1 | tee log.install - make distclean 2>&1 | tee log.distclean - ./configure \ - --prefix=/usr/local 2>&1 | tee log.config - make 2>&1 | tee log.make - make install 2>&1 | tee log.install - cd .. - rm -fr zlib-1.2.11 - # - tar -xvzf szip-2.1.1.tar.gz - cd szip-2.1.1 - ./configure \ - --prefix=/usr/local 2>&1 | tee log.config - make 2>&1 | tee log.make - make install 2>&1 | tee log.install - cd .. - rm -fr szip-2.1.1 - # - tar -xvjf hdf5-1.10.5.tar.bz2 - cd hdf5-1.10.5 - CC=mpicc \ - CXX=mpicxx \ - FC=mpif90 \ - ./configure \ - --prefix=/usr/local \ - --enable-parallel \ - --with-zlib=/usr/local \ - --with-szlib=/usr/local 2>&1 | tee log.config - make 2>&1 | tee log.make - make install 2>&1 | tee log.install - cd .. - rm -fr hdf5-1.10.5 - # - tar -xvzf netcdf-c-4.7.0.tar.gz - cd netcdf-c-4.7.0 - CC=mpicc \ - CXX=mpicxx \ - FC=mpif90 \ - ./configure \ - --prefix=/usr/local \ - --enable-parallel-tests 2>&1 | tee log.config - make 2>&1 | tee log.make - make install 2>&1 | tee log.install - cd .. - rm -fr netcdf-c-4.7.0 - # - tar -xvzf netcdf-fortran-4.4.5.tar.gz - cd netcdf-fortran-4.4.5 - CC=mpicc \ - CXX=mpicxx \ - FC=mpif90 \ - ./configure \ - --prefix=/usr/local \ - --enable-parallel-tests 2>&1 | tee log.config - make 2>&1 | tee log.make - make install 2>&1 | tee log.install - # - export NETCDF=/usr/local - - # NCEP libraries - git clone https://github.com/NCAR/NCEPlibs.git - mv NCEPlibs NCEPlibs-20190820 - cd NCEPlibs-20190820 - mkdir /usr/local/NCEPlibs-20190820 - ./make_ncep_libs.sh -s linux -c gnu -d /usr/local/NCEPlibs-20190820 -o 1 - cd .. - rm -fr NCEPlibs-20190820 - - export NCEPLIBS_DIR=/usr/local/NCEPlibs-20190820 - - # Download ESMF_8_0_0_beta_snapshot_50 from https://sourceforge.net/p/esmf/esmf/ref/master/tags/ - # to /home/ubuntu/src (creates a directory esmf-esmf-...), rename it to esmf-8.0.0_bs50 and tar it - # up for later use - cd /usr/local/src - mv esmf-esmf-... esmf-8.0.0_bs50 - tar -cvzf esmf-8.0.0_bs50.tar.gz esmf-8.0.0_bs50 - # - # Install esmf-8.0.0_bs50 - tar -xvzf esmf-8.0.0_bs50.tar.gz - cd esmf-8.0.0_bs50 - export ESMF_DIR=`pwd` - export ESMF_INSTALL_PREFIX=/usr/local/esmf-8.0.0_bs50 - export ESMF_CXXCOMPILER=mpicxx - export ESMF_CXXLINKER=mpicxx - export ESMF_F90COMPILER=mpif90 - export ESMF_F90LINKER=mpif90 - export ESMF_COMM=mpich3 - export ESMF_MPIRUN=mpiexec - export ESMF_NETCDF=nc-config - export ESMF_INSTALL_BINDIR=bin - export ESMF_INSTALL_LIBDIR=lib - export ESMF_INSTALL_MODDIR=mod - make info 2>&1 | tee log.info - make 2>&1 | tee log.make - # "make check" is optional and can take very long time - make check 2>&1 | tee log.check - # SYSTEM TESTS SUMMARY - # Found 45 multi-processor system tests, 44 passed and 1 failed. - # UNIT TESTS SUMMARY - # Found 3466 non-exhaustive multi-processor unit tests, 3466 passed and 0 failed. - # --> ignore and proceed - make install 2>&1 | tee log.install - make installcheck 2>&1 | tee log.installcheck - cd .. - rm -fr esmf-8.0.0_bs50 - export -n ESMF_DIR - export -n ESMF_INSTALL_PREFIX - export -n ESMF_CXXCOMPILER - export -n ESMF_CXXLINKER - export -n ESMF_F90COMPILER - export -n ESMF_F90LINKER - export -n ESMF_COMM - export -n ESMF_MPIRUN - export -n ESMF_NETCDF - export -n ESMF_INSTALL_BINDIR - export -n ESMF_INSTALL_LIBDIR - export -n ESMF_INSTALL_MODDIR - - export ESMFMKFILE=/usr/local/esmf-8.0.0_bs50/lib/esmf.mk - -4. Compile NEMSfv3gfs as normal user - - mkdir ~/scratch - - # clone this branch of NEMSfv3gfs to ~/scratch/NEMSfv3gfs - - cd ~/scratch/NEMSfv3gfs/tests - - export LD_LIBRARY_PATH="/usr/local/lib:/usr/local/lib64:$LD_LIBRARY_PATH" - export NCEPLIBS_DIR=/usr/local/NCEPlibs-20190820 - export NETCDF=/usr/local - export ESMFMKFILE=/usr/local/esmf-8.0.0_bs50/lib/esmf.mk - export CC=mpicc - export CXX=mpicxx - export F77=mpif77 - export F90=mpif90 - export FC=mpif90 - - ./compile.sh $PWD/../FV3 linux.gnu 'CCPP=N' 2>&1 | tee log.compile # without CCPP - ./compile.sh $PWD/../FV3 linux.gnu 'CCPP=Y' 2>&1 | tee log.compile # with CCPP, dynamic mode - ./compile.sh $PWD/../FV3 linux.gnu 'CCPP=Y STATIC=Y SUITES=FV3_GFS_2017_gfdlmp' 2>&1 | tee log.compile # with CCPP, static mode, GFS suite - -5. Set up the run directory using the template on Theia or Cheyenne at some location on your machine: - - a) copy the contents of the run directory templates to where you want to run the model, change to this directory - (these folders are read-only, i.e. users might have to add the write-flag after copying/rsyncing them) - - theia: /scratch4/BMC/gmtb/Dom.Heinzeller/rundirs/20190811/linux/fv3_gfdlmp/ - cheyenne: /glade/p/ral/jntp/GMTB/NEMSfv3gfs/rundirs/20190811/linux/fv3_gfdlmp/ - - b) edit run_linux.sh, set variables and change the variable FV3_BUILD_DIR to the top-level directory of your FV3-build - - c) set the environment variables as above (consider creating a shell script to source?) and run the model - - export NCEPLIBS_DIR=/usr/local/NCEPlibs-20190820 - export NETCDF=/usr/local - export ESMFMKFILE=/usr/local/esmf-8.0.0_bs50/lib/esmf.mk - - ./run_linux.sh 2>&1 | tee run_linux.log - # or, with N OpenMP threads (use N=1 for the dynamic CCPP build) - OMP_NUM_THREADS=N ./run_linux.sh 2>&1 | tee run_linux.log - - d) go and get yourself a cup of coffee ... diff --git a/doc/README_CHEYENNE.txt b/doc/README_CHEYENNE.txt deleted file mode 100644 index 333825d281..0000000000 --- a/doc/README_CHEYENNE.txt +++ /dev/null @@ -1,67 +0,0 @@ -################################################################################ -# Cheyenne/Intel -################################################################################ - -# OFFICIAL BASELINE -# /glade/p/ral/jntp/GMTB/NEMSfv3gfs/RT/trunk-20190912/INTEL - -# OWN BASELINE -# /glade/scratch/$USER/FV3_RT/REGRESSION_TEST_INTEL - -# RUN DIRS -# /glade/scratch/$USER/FV3_RT/rt_... - -export NEMS_COMPILER=intel # optional, default is intel if not set -export ACCNR=... # set account name/number for job scheduler - -# from the top-level directory of NEMSfv3gfs -cd tests - -# Test standard compile -./compile.sh $PWD/../FV3 cheyenne.intel 2>&1 | tee log.compile - -# Regression tests against official baseline -./rt.sh -f 2>&1 | tee rt.log - -# Create new baseline -./rt.sh -f -c 2>&1 | tee rt_create.log - -# Regression tests against new baseline -./rt.sh -f -m 2>&1 | tee rt_verify.log - -################################################################################ -# Cheyenne/GNU -################################################################################ - -# OFFICIAL BASELINE -# /glade/p/ral/jntp/GMTB/NEMSfv3gfs/RT/trunk-20190912/GNU - -# OWN BASELINE -# /glade/scratch/$USER/FV3_RT/REGRESSION_TEST_GNU - -# RUN DIRS -# /glade/scratch/$USER/FV3_RT/rt_... - -export NEMS_COMPILER=gnu -export ACCNR=... # set account name/number for job scheduler - -# from the top-level directory of NEMSfv3gfs -cd tests - -# Test standard compile -./compile.sh $PWD/../FV3 cheyenne.gnu 2>&1 | tee log.compile - -# Regression tests against official baseline -./rt.sh -l rt_gnu.conf 2>&1 | tee rt_gnu.log - -# Create new baseline -./rt.sh -l rt_gnu.conf -c 2>&1 | tee rt_gnu_create.log - -# Regression tests against new baseline -./rt.sh -l rt_gnu.conf -m 2>&1 | tee rt_gnu_verify.log - -################################################################################ -# Cheyenne/PGI -################################################################################ - -no longer supported diff --git a/doc/README_FV3GFS_Public_Release_FY18Q2 b/doc/README_FV3GFS_Public_Release_FY18Q2 deleted file mode 100644 index 5a7e260bbe..0000000000 --- a/doc/README_FV3GFS_Public_Release_FY18Q2 +++ /dev/null @@ -1,5 +0,0 @@ -This repository is a scientific product and is not official communication of the National Oceanic and Atmospheric Administration, or the United States Department of Commerce. All NOAA GitHub project code is provided on an ‘as is’ basis and the user assumes responsibility for its use. Any claims against the Department of Commerce or Department of Commerce bureaus stemming from the use of this GitHub project will be governed by all applicable Federal law. Any reference to specific commercial products, processes, or services by service mark, trademark, manufacturer, or otherwise, does not constitute or imply their endorsement, recommendation or -favoring by the Department of Commerce. The Department of Commerce seal and logo, or the seal and logo of a DOC bureau, shall not be used in any manner to imply endorsement of any commercial product or activity by DOC or the United States Government. - -The code is supported on NOAA research and development machines THEIA, GAEA and Jet, where prebuilt libraries and test cases are available. Best efforts will be provided to support users to run this public release code on those machines. - diff --git a/doc/README_HERA.txt b/doc/README_HERA.txt deleted file mode 100644 index bf42209187..0000000000 --- a/doc/README_HERA.txt +++ /dev/null @@ -1,80 +0,0 @@ -################################################################################ -# Hera/Intel -################################################################################ - -# OFFICIAL BASELINE -# /scratch1/NCEPDEV/nems/emc.nemspara/RT/NEMSfv3gfs/trunk-20190824 - -# OWN BASELINE -# /scratch1/NCEPDEV/stmp4/$USER/FV3_RT - -# RUN DIRS - substitute $USER with your username -# /scratch1/NCEPDEV/stmp2/$USER/FV3_RT/rt_... - -export NEMS_COMPILER=intel # optional, default is intel if not set -export ACCNR=... # set account name/number for job scheduler - -# from the top-level directory of NEMSfv3gfs -cd tests - -# Test standard compile -./compile.sh $PWD/../FV3 hera.intel 2>&1 | tee log.compile - -# Regression tests against official baseline -./rt.sh -f 2>&1 | tee rt_full.log - -################################################################################ -# Hera/GNU -################################################################################ - -# OFFICIAL BASELINE -# N/A - -# OWN BASELINE -# /scratch1/NCEPDEV/stmp4/$USER/FV3_RT/REGRESSION_TEST_GNU - -# RUN DIRS -# /scratch1/NCEPDEV/stmp2/$USER/FV3_RT/rt_... - -export NEMS_COMPILER=gnu -export ACCNR=... # set account name/number for job scheduler - -# from the top-level directory of NEMSfv3gfs -cd tests - -# Test standard compile -./compile.sh $PWD/../FV3 hera.gnu 2>&1 | tee log.compile - -# Create new baseline -./rt.sh -f -l rt_gnu_pgi.conf -c fv3 2>&1 | tee rt_create_sub.log - -# Regression tests against new baseline -./rt.sh -f -l rt_gnu_pgi.conf -m 2>&1 | tee rt_sub.log - -################################################################################ -# Hera/PGI -################################################################################ - -# OFFICIAL BASELINE -# N/A - -# OWN BASELINE -# /scratch1/NCEPDEV/stmp4/$USER/FV3_RT/REGRESSION_TEST_PGI - -# RUN DIRS -# /scratch1/NCEPDEV/stmp2/$USER/FV3_RT/rt_... - -export NEMS_COMPILER=pgi -export ACCNR=... # set account name/number for job scheduler - -# from the top-level directory of NEMSfv3gfs -cd tests - -# Test standard compile -./compile.sh $PWD/../FV3 hera.pgi 2>&1 | tee log.compile - -# Create new baseline -./rt.sh -f -l rt_gnu_pgi.conf -c fv3 2>&1 | tee rt_create_sub.log - -# Regression tests against new baseline -./rt.sh -f -l rt_gnu_pgi.conf -m 2>&1 | tee rt_sub.log diff --git a/doc/README_MACOSX_clanggfortran.txt b/doc/README_MACOSX_clanggfortran.txt deleted file mode 100644 index b6effc1b6d..0000000000 --- a/doc/README_MACOSX_clanggfortran.txt +++ /dev/null @@ -1,229 +0,0 @@ -# Dom Heinzeller (dom.heinzeller@noaa.gov), 08/12/2019 - -Target systems: macOS High Sierra / macOS Mojave with LLVM clang + GNU gfortran compilers and mpich MPI library. - -In order to build and run the FV3 trunk (August 2019) with possible CCPP extensions by GMTB on macOS, -the following installation steps are recommended. The version numbers for the "brew" correspond to the default versions -in August 2019 and will change to newer versions in the future. Unless problems occur during the manual builds in -steps 6-11, these differences can be ignored. It is also assumed that the bash shell is used in the following. - -1. Install homebrew (enter sudo password when requested) - /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)" - -2. Create /usr/local/src, /usr/local/esmf-8.0.0_bs50 and /usr/local/NCEPlibs-20190811. Change permissions to your user name / user group - - # change to root mode - sudo su - # /usr/local/src - mkdir /usr/local/src - chown YOUR_USERNAME /usr/local/src - chgrp YOUR_GROUPNAME /usr/local/src - # /usr/local/esmf-8.0.0_bs50 - mkdir /usr/local/esmf-8.0.0_bs50 - chown YOUR_USERNAME /usr/local/esmf-8.0.0_bs50 - chgrp YOUR_GROUPNAME /usr/local/esmf-8.0.0_bs50 - # /usr/local/NCEPlibs-20190811 - mkdir /usr/local/NCEPlibs-20190811 - chown YOUR_USERNAME /usr/local/NCEPlibs-20190811 - chgrp YOUR_GROUPNAME /usr/local/NCEPlibs-20190811 - # leave root mode - exit - -3. Use homebrew to install the following packages - - a) Install gcc-9.1.0, gfortran-9.1.0 - brew install gcc@9 - - b) Install clang-8.0.1 with openmp support - brew install llvm - - c) Install mpich-3.3.1 - brew install mpich - - d) Install netcdf-4.6.3 - brew install netcdf - - e) Install libpng-1.6.37 - brew install libpng - - f) Install udunits-2.2.27 - brew install udunits - - g) Install cmake-3.15.2 - brew install cmake - - h) Install xquartz-2.7.11 (optional, only needed for ncview) - brew cask install xquartz - - i) Install ncview-2.1.7 (optional) - brew install ncview - -4. Create a shell setup script ~/setenv_develop_nemsfv3gfs.sh to set the required paths for compiling NEMSfv3gfs. Contents: - -#################################################################################### -#!/bin/bash - -echo "Setting environment variables for develop-NEMSfv3gfs" - -export PATH="/usr/local/opt/llvm/bin:$PATH" -export CPPFLAGS="-I/usr/local/opt/llvm/include" -export LDFLAGS="-L/usr/local/opt/llvm/lib -Wl,-rpath,/usr/local/opt/llvm/lib" -export LIBS_OPENMP="-L/usr/local/opt/llvm/lib -lomp" - -export CC=/usr/local/opt/llvm/bin/clang -export CXX=/usr/local/opt/llvm/bin/clang++ -export FC=/usr/local/bin/gfortran -export F90=/usr/local/bin/gfortran -export F77=/usr/local/bin/gfortran - -export MPICC="mpicc -cc=${CC}" -export MPICXX="mpicxx -cxx=${CXX}" -export MPIFORT=/usr/local/bin/mpifort -export MPIF77=/usr/local/bin/mpif77 -export MPIF90=/usr/local/bin/mpif90 - -export HDF5=/usr/local -export NETCDF=/usr/local -export ESMFMKFILE=/usr/local/esmf-8.0.0_bs50/lib/esmf.mk -export NCEPLIBS_DIR=/usr/local/NCEPlibs-20190811 -export MKL_DIR=/opt/intel/compilers_and_libraries_2019.4.233/mac/mkl - -#################################################################################### - -5. Source this shell script for compiling the libraries and the model below - . ~/setenv_develop_nemsfv3gfs.sh - -6. Install ESMF 8.0.0_bs50 - - # Download ESMF_8_0_0_beta_snapshot_50 from https://sourceforge.net/p/esmf/esmf/ref/master/tags/ - # to /usr/local/src (creates a directory esmf-esmf-...), rename it to esmf-8.0.0_bs50 and tar it - # up for later use - cd /usr/local/src - mv esmf-esmf-... esmf-8.0.0_bs50 - tar -cvzf esmf-8.0.0_bs50.tar.gz esmf-8.0.0_bs50 - - # Compile and install ESMF - cd esmf-8.0.0_bs50 - export ESMF_DIR=`pwd` - export ESMF_COMPILER=gfortranclang - export ESMF_CXXCOMPILER=$MPICXX - export ESMF_CXXLINKER=$MPICXX - export ESMF_F90COMPILER=$MPIF90 - export ESMF_F90LINKER=$MPIF90 - export ESMF_BOPT=O - export ESMF_OPTLEVEL=2 - export ESMF_COMM=mpich - export ESMF_MPIRUN=mpirun - export ESMF_NETCDF=1 - export ESMF_NETCDF_INCLUDE=$NETCDF/include - export ESMF_NETCDF_LIBPATH=$NETCDF/lib - export ESMF_NETCDF_LIBS="-lnetcdff -lnetcdf -lmpichf90" - export ESMF_NETCDF=split - export ESMF_INSTALL_PREFIX=/usr/local/esmf-8.0.0_bs50 - export ESMF_INSTALL_BINDIR=bin - export ESMF_INSTALL_LIBDIR=lib - export ESMF_INSTALL_MODDIR=mod - # - make info 2>&1 | tee log.info - make 2>&1 | tee log.make - # "make check" is optional and can take very long time - make check 2>&1 | tee log.check - # SYSTEM TESTS SUMMARY - # Found 45 multi-processor system tests, 38 passed and 7 failed. - # UNIT TESTS SUMMARY - # Found 3466 non-exhaustive multi-processor unit tests, 3394 passed and 72 failed. - # --> ignore and proceed - make install 2>&1 | tee log.install - make installcheck 2>&1 | tee log.installcheck - # - cd $ESMF_INSTALL_PREFIX - # - # Fix wrong path to libesmf.dylib in ESMF binaries - this will hopefully be addressed in future ESMF releases - # - install_name_tool -change $ESMF_DIR/lib/libO/Darwin.gfortranclang.64.mpich.default/libesmf.dylib $ESMF_INSTALL_PREFIX/lib/libesmf.dylib bin/ESMF_Info - install_name_tool -change $ESMF_DIR/lib/libO/Darwin.gfortranclang.64.mpich.default/libesmf.dylib $ESMF_INSTALL_PREFIX/lib/libesmf.dylib bin/ESMF_InfoC - install_name_tool -change $ESMF_DIR/lib/libO/Darwin.gfortranclang.64.mpich.default/libesmf.dylib $ESMF_INSTALL_PREFIX/lib/libesmf.dylib bin/ESMF_WebServController - install_name_tool -change $ESMF_DIR/lib/libO/Darwin.gfortranclang.64.mpich.default/libesmf.dylib $ESMF_INSTALL_PREFIX/lib/libesmf.dylib bin/ESMF_Regrid - install_name_tool -change $ESMF_DIR/lib/libO/Darwin.gfortranclang.64.mpich.default/libesmf.dylib $ESMF_INSTALL_PREFIX/lib/libesmf.dylib bin/ESMF_RegridWeightGen - install_name_tool -change $ESMF_DIR/lib/libO/Darwin.gfortranclang.64.mpich.default/libesmf.dylib $ESMF_INSTALL_PREFIX/lib/libesmf.dylib bin/ESMF_Scrip2Unstruct - # - # Fix wrong ID in libesmf.dylib - this will hopefully be addressed in future ESMF releases - # - install_name_tool -id $ESMF_INSTALL_PREFIX/lib/libesmf.dylib lib/libesmf.dylib - install_name_tool -id $ESMF_INSTALL_PREFIX/lib/libesmf_fullylinked.dylib lib/libesmf_fullylinked.dylib - # - # Clean up - cd /usr/local/src - rm -fr esmf-8.0.0_bs50 - export -n ESMF_DIR - export -n ESMF_COMPILER - export -n ESMF_CXXCOMPILER - export -n ESMF_CXXLINKER - export -n ESMF_F90COMPILER - export -n ESMF_F90LINKER - export -n ESMF_BOPT - export -n ESMF_OPTLEVEL - export -n ESMF_COMM - export -n ESMF_MPIRUN - export -n ESMF_NETCDF - export -n ESMF_NETCDF_INCLUDE - export -n ESMF_NETCDF_LIBPATH - export -n ESMF_NETCDF_LIBS - export -n ESMF_NETCDF - export -n ESMF_INSTALL_PREFIX - export -n ESMF_INSTALL_BINDIR - export -n ESMF_INSTALL_LIBDIR - export -n ESMF_INSTALL_MODDIR - -7. Build external NCEP libraries (use date tag 20190811 to allow for different versions in the future) - - # Obtain source code from gitub and build in /usr/local/src - cd /usr/local/src - git clone https://github.com/NCAR/NCEPlibs.git NCEPlibs-20190811 - cd NCEPlibs-20190811 - # Requires exporting CC, F90, MPIF90 (done by setenv_develop_nemsfv3gfs.sh) - ./make_ncep_libs.sh -s macosx -c gnu -d /usr/local/NCEPlibs-20190811 -o 1 2>&1 | tee log.make - -8. Download and install Intel Math Kernel Library MKL (full package) from https://software.intel.com/en-us/mkl - to /opt/intel (default location) using the installer script (requires sudo/root access) - -9. Pro-tip (optional): have your computer remember your GitHub username and password. - See https://help.github.com/en/articles/caching-your-github-password-in-git. - - git config --global credential.helper osxkeychain - -10. Download the model from GitHub: - cd $HOME - mkdir NEMSfv3gfs - cd NEMSfv3gfs - git clone --branch=gmtb/develop https://github.com/NCAR/NEMSfv3gfs NEMSfv3gfs-gmtb-develop-20190811 - cd NEMSfv3gfs-gmtb-develop-20190811 - git submodule init - git submodule update - -11. Build model. Change to top-level directory of NEMSfv3gfs-gmtb-develop-20190811 - - . ~/setenv_develop_nemsfv3gfs.sh - cd tests - # Note: omit '32BIT=Y' to compile dynamics in double precision (slower to run) - ./compile.sh $PWD/../FV3 macosx.gnu '32BIT=Y CCPP=N' 2>&1 | tee log.compile # without CCPP - ./compile.sh $PWD/../FV3 macosx.gnu '32BIT=Y CCPP=Y' 2>&1 | tee log.compile # with CCPP, dynamic mode - ./compile.sh $PWD/../FV3 macosx.gnu '32BIT=Y CCPP=Y STATIC=Y SUITES=FV3_GFS_2017_gfdlmp' 2>&1 | tee log.compile # with CCPP, static mode, GFS suite - -12. Set up the run directory using the template on Theia or Cheyenne at some location on your machine: - - a) copy the contents of the run directory templates to where you want to run the model, change to this directory - (these folders are read-only, i.e. users might have to add the write-flag after copying/rsyncing them) - - theia: /scratch4/BMC/gmtb/Dom.Heinzeller/rundirs/20190811/macosx/fv3_gfdlmp/ - cheyenne: /glade/p/ral/jntp/GMTB/NEMSfv3gfs/rundirs/20190811/macosx/fv3_gfdlmp/ - - b) edit run_macosx.sh, set variables and change the variable FV3_BUILD_DIR to the top-level directory of your FV3-build - - c) source ~/setenv_develop_nemsfv3gfs.sh and execute the model run using the wrapper run_macosx.sh - . ~/setenv_develop_nemsfv3gfs.sh - ./run_macosx.sh 2>&1 | tee run_macosx.log - # or, with N OpenMP threads (use N=1 for the dynamic CCPP build) - OMP_NUM_THREADS=N ./run_macosx.sh 2>&1 | tee run_macosx.log - - d) go and get yourself a cup of coffee ... diff --git a/doc/README_MACOSX_gccgfortran.txt b/doc/README_MACOSX_gccgfortran.txt deleted file mode 100644 index 7d7b81e791..0000000000 --- a/doc/README_MACOSX_gccgfortran.txt +++ /dev/null @@ -1,221 +0,0 @@ -# Dom Heinzeller (dom.heinzeller@noaa.gov), 08/12/2019 - -Target systems: macOS High Sierra / macOS Mojave with GNU gcc+gfortran compilers and mpich MPI library. - -In order to build and run the FV3 trunk (August 2019) with possible CCPP extensions by GMTB on macOS, -the following installation steps are recommended. The version numbers for the "brew" correspond to the default versions -in August 2019 and will change to newer versions in the future. Unless problems occur during the manual builds in -steps 6-11, these differences can be ignored. It is also assumed that the bash shell is used in the following. - -1. Install homebrew (enter sudo password when requested) - /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)" - -2. Create /usr/local/src, /usr/local/esmf-8.0.0_bs50 and /usr/local/NCEPlibs-20190811. Change permissions to your user name / user group - - # change to root mode - sudo su - # /usr/local/src - mkdir /usr/local/src - chown YOUR_USERNAME /usr/local/src - chgrp YOUR_GROUPNAME /usr/local/src - # /usr/local/esmf-8.0.0_bs50 - mkdir /usr/local/esmf-8.0.0_bs50 - chown YOUR_USERNAME /usr/local/esmf-8.0.0_bs50 - chgrp YOUR_GROUPNAME /usr/local/esmf-8.0.0_bs50 - # /usr/local/NCEPlibs-20190811 - mkdir /usr/local/NCEPlibs-20190811 - chown YOUR_USERNAME /usr/local/NCEPlibs-20190811 - chgrp YOUR_GROUPNAME /usr/local/NCEPlibs-20190811 - # leave root mode - exit - -3. Use homebrew to install the following packages - - a) Install gcc-8.3.0, gfortran-8.3.0 - brew install gcc@8 - - b) Install mpich-3.3.1 - brew install mpich - - c) Install netcdf-4.6.3 - brew install netcdf - - d) Install libpng-1.6.37 - brew install libpng - - e) Install udunits-2.2.27 - brew install udunits - - f) Install cmake-3.15.2 - brew install cmake - - g) Install xquartz-2.7.11 (optional, only needed for ncview) - brew cask install xquartz - - h) Install ncview-2.1.7 (optional) - brew install ncview - -4. Create a shell setup script ~/setenv_develop_nemsfv3gfs.sh to set the required paths for compiling NEMSfv3gfs. Contents: - -#################################################################################### -#!/bin/bash - -echo "Setting environment variables for develop-NEMSfv3gfs" - -export CC=gcc-8 -export CXX=g++-8 -export FC=gfortran-8 -export F90=gfortran-8 -export F77=gfortran-8 - -export MPICC="mpicc -cc=$CC" -export MPICXX="mpicxx -cxx=$CXX" -export MPIFORT="mpifort -fc=$FC" -export MPIF77="mpif77 -fc=$F77" -export MPIF90="mpif90 -fc=$F90" - -export HDF5=/usr/local -export NETCDF=/usr/local -export ESMFMKFILE=/usr/local/esmf-8.0.0_bs50/lib/esmf.mk -export NCEPLIBS_DIR=/usr/local/NCEPlibs-20190811 -export MKL_DIR=/opt/intel/compilers_and_libraries_2019.4.233/mac/mkl - -#################################################################################### - -5. Source this shell script for compiling the libraries and the model below - . ~/setenv_develop_nemsfv3gfs.sh - -6. Install ESMF 8.0.0_bs50 - - # Download ESMF_8_0_0_beta_snapshot_50 from https://sourceforge.net/p/esmf/esmf/ref/master/tags/ - # to /usr/local/src (creates a directory esmf-esmf-...), rename it to esmf-8.0.0_bs50 and tar it - # up for later use - cd /usr/local/src - mv esmf-esmf-... esmf-8.0.0_bs50 - tar -cvzf esmf-8.0.0_bs50.tar.gz esmf-8.0.0_bs50 - - # Compile and install ESMF - cd esmf-8.0.0_bs50 - export ESMF_DIR=`pwd` - export ESMF_COMPILER=gfortran - export ESMF_CXXCOMPILER=$MPICXX - export ESMF_CXXLINKER=$MPICXX - export ESMF_F90COMPILER=$MPIF90 - export ESMF_F90LINKER=$MPIF90 - export ESMF_BOPT=O - export ESMF_OPTLEVEL=2 - export ESMF_COMM=mpich - export ESMF_MPIRUN=mpirun - export ESMF_NETCDF=1 - export ESMF_NETCDF_INCLUDE=$NETCDF/include - export ESMF_NETCDF_LIBPATH=$NETCDF/lib - export ESMF_NETCDF_LIBS="-lnetcdff -lnetcdf -lmpichf90" - export ESMF_NETCDF=split - export ESMF_INSTALL_PREFIX=/usr/local/esmf-8.0.0_bs50 - export ESMF_INSTALL_BINDIR=bin - export ESMF_INSTALL_LIBDIR=lib - export ESMF_INSTALL_MODDIR=mod - # - make info 2>&1 | tee log.info - make 2>&1 | tee log.make - # "make check" is optional and can take very long time - make check 2>&1 | tee log.check - # SYSTEM TESTS SUMMARY - # Found 45 multi-processor system tests, 38 passed and 7 failed. - # UNIT TESTS SUMMARY - # Found 3466 non-exhaustive multi-processor unit tests, 3394 passed and 72 failed. - # --> ignore and proceed - make install 2>&1 | tee log.install - make installcheck 2>&1 | tee log.installcheck - # - cd $ESMF_INSTALL_PREFIX - # - # Fix wrong path to libesmf.dylib in ESMF binaries - this will hopefully be addressed in future ESMF releases - # - install_name_tool -change $ESMF_DIR/lib/libO/Darwin.gfortran.64.mpich.default/libesmf.dylib $ESMF_INSTALL_PREFIX/lib/libesmf.dylib bin/ESMF_Info - install_name_tool -change $ESMF_DIR/lib/libO/Darwin.gfortran.64.mpich.default/libesmf.dylib $ESMF_INSTALL_PREFIX/lib/libesmf.dylib bin/ESMF_InfoC - install_name_tool -change $ESMF_DIR/lib/libO/Darwin.gfortran.64.mpich.default/libesmf.dylib $ESMF_INSTALL_PREFIX/lib/libesmf.dylib bin/ESMF_WebServController - install_name_tool -change $ESMF_DIR/lib/libO/Darwin.gfortran.64.mpich.default/libesmf.dylib $ESMF_INSTALL_PREFIX/lib/libesmf.dylib bin/ESMF_Regrid - install_name_tool -change $ESMF_DIR/lib/libO/Darwin.gfortran.64.mpich.default/libesmf.dylib $ESMF_INSTALL_PREFIX/lib/libesmf.dylib bin/ESMF_RegridWeightGen - install_name_tool -change $ESMF_DIR/lib/libO/Darwin.gfortran.64.mpich.default/libesmf.dylib $ESMF_INSTALL_PREFIX/lib/libesmf.dylib bin/ESMF_Scrip2Unstruct - # - # Fix wrong ID in libesmf.dylib - this will hopefully be addressed in future ESMF releases - # - install_name_tool -id $ESMF_INSTALL_PREFIX/lib/libesmf.dylib lib/libesmf.dylib - install_name_tool -id $ESMF_INSTALL_PREFIX/lib/libesmf_fullylinked.dylib lib/libesmf_fullylinked.dylib - # - # Clean up - cd /usr/local/src - rm -fr esmf-8.0.0_bs50 - export -n ESMF_DIR - export -n ESMF_COMPILER - export -n ESMF_CXXCOMPILER - export -n ESMF_CXXLINKER - export -n ESMF_F90COMPILER - export -n ESMF_F90LINKER - export -n ESMF_BOPT - export -n ESMF_OPTLEVEL - export -n ESMF_COMM - export -n ESMF_MPIRUN - export -n ESMF_NETCDF - export -n ESMF_NETCDF_INCLUDE - export -n ESMF_NETCDF_LIBPATH - export -n ESMF_NETCDF_LIBS - export -n ESMF_NETCDF - export -n ESMF_INSTALL_PREFIX - export -n ESMF_INSTALL_BINDIR - export -n ESMF_INSTALL_LIBDIR - export -n ESMF_INSTALL_MODDIR - -7. Build external NCEP libraries (use date tag 20190811 to allow for different versions in the future) - - # Obtain source code from gitub and build in /usr/local/src - cd /usr/local/src - git clone https://github.com/NCAR/NCEPlibs.git NCEPlibs-20190811 - cd NCEPlibs-20190811 - # Requires exporting CC, F90, MPIF90 (done by setenv_develop_nemsfv3gfs.sh) - ./make_ncep_libs.sh -s macosx -c gnu -d /usr/local/NCEPlibs-20190811 -o 1 2>&1 | tee log.make - -8. Download and install Intel Math Kernel Library MKL (full package) from https://software.intel.com/en-us/mkl - to /opt/intel (default location) using the installer script (requires sudo/root access) - -9. Pro-tip (optional): have your computer remember your GitHub username and password. - See https://help.github.com/en/articles/caching-your-github-password-in-git. - - git config --global credential.helper osxkeychain - -10. Download the model from GitHub: - cd $HOME - mkdir NEMSfv3gfs - cd NEMSfv3gfs - git clone --branch=gmtb/develop https://github.com/NCAR/NEMSfv3gfs NEMSfv3gfs-gmtb-develop-20190811 - cd NEMSfv3gfs-gmtb-develop-20190811 - git submodule init - git submodule update - -11. Build model. Change to top-level directory of NEMSfv3gfs-gmtb-develop-20190811 - - . ~/setenv_develop_nemsfv3gfs.sh - cd tests - # Note: omit '32BIT=Y' to compile dynamics in double precision (slower to run) - ./compile.sh $PWD/../FV3 macosx.gnu '32BIT=Y CCPP=N' 2>&1 | tee log.compile # without CCPP - ./compile.sh $PWD/../FV3 macosx.gnu '32BIT=Y CCPP=Y' 2>&1 | tee log.compile # with CCPP, dynamic mode - ./compile.sh $PWD/../FV3 macosx.gnu '32BIT=Y CCPP=Y STATIC=Y SUITES=FV3_GFS_2017_gfdlmp' 2>&1 | tee log.compile # with CCPP, static mode, GFS suite - -12. Set up the run directory using the template on Theia or Cheyenne at some location on your machine: - - a) copy the contents of the run directory templates to where you want to run the model, change to this directory - (these folders are read-only, i.e. users might have to add the write-flag after copying/rsyncing them) - - theia: /scratch4/BMC/gmtb/Dom.Heinzeller/rundirs/20190811/macosx/fv3_gfdlmp/ - cheyenne: /glade/p/ral/jntp/GMTB/NEMSfv3gfs/rundirs/20190811/macosx/fv3_gfdlmp/ - - b) edit run_macosx.sh, set variables and change the variable FV3_BUILD_DIR to the top-level directory of your FV3-build - - c) source ~/setenv_develop_nemsfv3gfs.sh and execute the model run using the wrapper run_macosx.sh - . ~/setenv_develop_nemsfv3gfs.sh - ./run_macosx.sh 2>&1 | tee run_macosx.log - # or, with N OpenMP threads (use N=1 for the dynamic CCPP build) - OMP_NUM_THREADS=N ./run_macosx.sh 2>&1 | tee run_macosx.log - - d) go and get yourself a cup of coffee ... diff --git a/doc/README_THEIA.txt b/doc/README_THEIA.txt deleted file mode 100644 index c58d815a4c..0000000000 --- a/doc/README_THEIA.txt +++ /dev/null @@ -1,80 +0,0 @@ -################################################################################ -# Theia/Intel -################################################################################ - -# OFFICIAL BASELINE -# /scratch4/NCEPDEV/nems/noscrub/emc.nemspara/RT/NEMSfv3gfs/trunk-20190315 - -# OWN BASELINE -# /scratch4/NCEPDEV/stmp4/$USER/FV3_RT/REGRESSION_TEST_INTEL - -# RUN DIRS - substitute $USER with your username -# /scratch4/NCEPDEV/stmp3/$USER/FV3_RT/rt_... - -export NEMS_COMPILER=intel # optional, default is intel if not set -export ACCNR=... # set account name/number for job scheduler - -# from the top-level directory of NEMSfv3gfs -cd tests - -# Test standard compile -./compile.sh $PWD/../FV3 theia.intel 2>&1 | tee log.compile - -# Regression tests against official baseline -./rt.sh -f 2>&1 | tee rt_full.log - -################################################################################ -# Theia/GNU -################################################################################ - -# OFFICIAL BASELINE -# N/A - -# OWN BASELINE -# /scratch4/NCEPDEV/stmp4/$USER/FV3_RT/REGRESSION_TEST_GNU - -# RUN DIRS -# /scratch4/NCEPDEV/stmp3/$USER/FV3_RT/rt_... - -export NEMS_COMPILER=gnu -export ACCNR=... # set account name/number for job scheduler - -# from the top-level directory of NEMSfv3gfs -cd tests - -# Test standard compile -./compile.sh $PWD/../FV3 theia.gnu 2>&1 | tee log.compile - -# Create new baseline -./rt.sh -f -l rt_gnu_pgi.conf -c fv3 2>&1 | tee rt_create_sub.log - -# Regression tests against new baseline -./rt.sh -f -l rt_gnu_pgi.conf -m 2>&1 | tee rt_sub.log - -################################################################################ -# Theia/PGI -################################################################################ - -# OFFICIAL BASELINE -# N/A - -# OWN BASELINE -# /scratch4/NCEPDEV/stmp4/$USER/FV3_RT/REGRESSION_TEST_PGI - -# RUN DIRS -# /scratch4/NCEPDEV/stmp3/$USER/FV3_RT/rt_... - -export NEMS_COMPILER=pgi -export ACCNR=... # set account name/number for job scheduler - -# from the top-level directory of NEMSfv3gfs -cd tests - -# Test standard compile -./compile.sh $PWD/../FV3 theia.pgi 2>&1 | tee log.compile - -# Create new baseline -./rt.sh -f -l rt_gnu_pgi.conf -c fv3 2>&1 | tee rt_create_sub.log - -# Regression tests against new baseline -./rt.sh -f -l rt_gnu_pgi.conf -m 2>&1 | tee rt_sub.log diff --git a/doc/README_UBUNTU.txt b/doc/README_UBUNTU.txt deleted file mode 100644 index 7e4ead9fbf..0000000000 --- a/doc/README_UBUNTU.txt +++ /dev/null @@ -1,135 +0,0 @@ -# Dom Heinzeller (dom.heinzeller@noaa.gov), 08/21/2019 - -In order to build and run the FV3 trunk (August 2019) with possible CCPP extensions by GMTB on Ubuntu Linux, -the following installation steps are recommended. The version numbers correspond to the default versions in -August 2019 and will change to newer versions in the future. Unless problems occur during the manual builds in -step 4, these differences can be ignored. It is also assumed that the bash shell is used in the following. - -1. Install Ubuntu Linux 18.04.1 LTS or start up cloud instance (e.g. Amazon AWS) with Ubuntu Linux 18.04.1 LTS - -2. Install standard packages as root (sudo su) - - apt update - apt install ssh - apt install gfortran libgfortran-7-dev - apt install g++ libstdc++-7-dev - apt install make - apt install cmake - apt install m4 - apt install ksh - apt install git - apt install python - apt install libxml2-dev - apt install libnetcdff-dev - apt install mpich - ln -s /usr/bin/make /usr/bin/gmake - - export NETCDF=/usr - -3. Install thirdparty libraries - - mkdir -p /usr/local/src && cd /usr/local/src - - # NCEP libraries - git clone https://github.com/NCAR/NCEPlibs.git - mv NCEPlibs NCEPlibs-20190820 - cd NCEPlibs-20190820 - mkdir /usr/local/NCEPlibs-20190820 - ./make_ncep_libs.sh -s linux -c gnu -d /usr/local/NCEPlibs-20190820 -o 1 - cd .. - rm -fr NCEPlibs-20190820 - - export NCEPLIBS_DIR=/usr/local/NCEPlibs-20190820 - - # Download ESMF_8_0_0_beta_snapshot_50 from https://sourceforge.net/p/esmf/esmf/ref/master/tags/ - # to /home/ubuntu/src (creates a directory esmf-esmf-...), rename it to esmf-8.0.0_bs50 and tar it - # up for later use - cd /usr/local/src - mv esmf-esmf-... esmf-8.0.0_bs50 - tar -cvzf esmf-8.0.0_bs50.tar.gz esmf-8.0.0_bs50 - # - # Install esmf-8.0.0_bs50 - tar -xvzf esmf-8.0.0_bs50.tar.gz - cd esmf-8.0.0_bs50 - export ESMF_DIR=`pwd` - export ESMF_INSTALL_PREFIX=/usr/local/esmf-8.0.0_bs50 - export ESMF_CXXCOMPILER=mpicxx - export ESMF_CXXLINKER=mpicxx - export ESMF_F90COMPILER=mpif90 - export ESMF_F90LINKER=mpif90 - export ESMF_COMM=mpich3 - export ESMF_MPIRUN=mpiexec - export ESMF_NETCDF=nc-config - export ESMF_INSTALL_BINDIR=bin - export ESMF_INSTALL_LIBDIR=lib - export ESMF_INSTALL_MODDIR=mod - make info 2>&1 | tee log.info - make 2>&1 | tee log.make - # "make check" is optional and can take very long time - make check 2>&1 | tee log.check - # SYSTEM TESTS SUMMARY - # Found 45 multi-processor system tests, 45 passed and 0 failed. - # UNIT TESTS SUMMARY - # Found 3466 non-exhaustive multi-processor unit tests, 3464 passed and 2 failed. - # --> ignore and proceed - make install 2>&1 | tee log.install - make installcheck 2>&1 | tee log.installcheck - cd .. - rm -fr esmf-8.0.0_bs50 - export -n ESMF_DIR - export -n ESMF_INSTALL_PREFIX - export -n ESMF_CXXCOMPILER - export -n ESMF_CXXLINKER - export -n ESMF_F90COMPILER - export -n ESMF_F90LINKER - export -n ESMF_COMM - export -n ESMF_MPIRUN - export -n ESMF_NETCDF - export -n ESMF_INSTALL_BINDIR - export -n ESMF_INSTALL_LIBDIR - export -n ESMF_INSTALL_MODDIR - - export ESMFMKFILE=/usr/local/esmf-8.0.0_bs50/lib/esmf.mk - -4. Compile NEMSfv3gfs as normal user - - mkdir ~/scratch - - # clone this branch of NEMSfv3gfs to ~/scratch/NEMSfv3gfs - - cd ~/scratch/NEMSfv3gfs/tests - - export NCEPLIBS_DIR=/usr/local/NCEPlibs-20190820 - export NETCDF=/usr - export ESMFMKFILE=/usr/local/esmf-8.0.0_bs50/lib/esmf.mk - export CC=mpicc - export CXX=mpicxx - export F77=mpif77 - export F90=mpif90 - export FC=mpif90 - - ./compile.sh $PWD/../FV3 linux.gnu 'CCPP=N' 2>&1 | tee log.compile # without CCPP - ./compile.sh $PWD/../FV3 linux.gnu 'CCPP=Y' 2>&1 | tee log.compile # with CCPP, dynamic mode - ./compile.sh $PWD/../FV3 linux.gnu 'CCPP=Y STATIC=Y SUITES=FV3_GFS_2017_gfdlmp' 2>&1 | tee log.compile # with CCPP, static mode, GFS suite - -5. Set up the run directory using the template on Theia or Cheyenne at some location on your machine: - - a) copy the contents of the run directory templates to where you want to run the model, change to this directory - (these folders are read-only, i.e. users might have to add the write-flag after copying/rsyncing them) - - theia: /scratch4/BMC/gmtb/Dom.Heinzeller/rundirs/20190811/linux/fv3_gfdlmp/ - cheyenne: /glade/p/ral/jntp/GMTB/NEMSfv3gfs/rundirs/20190811/linux/fv3_gfdlmp/ - - b) edit run_linux.sh, set variables and change the variable FV3_BUILD_DIR to the top-level directory of your FV3-build - - c) set the environment variables as above (consider creating a shell script to source?) and run the model - - export NCEPLIBS_DIR=/usr/local/NCEPlibs-20190820 - export NETCDF=/usr - export ESMFMKFILE=/usr/local/esmf-8.0.0_bs50/lib/esmf.mk - - ./run_linux.sh 2>&1 | tee run_linux.log - # or, with N OpenMP threads (use N=1 for the dynamic CCPP build) - OMP_NUM_THREADS=N ./run_linux.sh 2>&1 | tee run_linux.log - - d) go and get yourself a cup of coffee ...