Parallel netcdf-4 IO option by MicroTed · Pull Request #1552 · wrf-model/WRF

MicroTed · 2021-09-09T00:55:28Z

TYPE: new feature

KEYWORDS: parallel netcdf4

SOURCE: Ted Mansell (NOAA/National Severe Storms Lab)

DESCRIPTION OF CHANGES:
This PR adds an I/O option to use the new netcdf capability in 4.7.4 (and later versions) to write both in parallel
and with variable compression for the same file. Versions of pnetcdf permitted parallel I/O, but did not permit
standard HDF5 compression. A new I/O library directory, external/io_netcdfpar, has been added that is
modeled on the similar external/io_netcdf directory. Users should see the file doc/README.netcdf4par for
details on configuring and running.

In a nutshell, steps to use this feature:

This capability to use HDF5 compression and to have parallel I/O (similar to pnetcdf) requires NetCDF v4.7.4
or later.
The NetCDF library must be built with MPI.
Once the NetCDF library / module is chosen and the NETCDF env variable is correctly pointing to the right
location, then prior to the configure step, the user sets a new environment variable. For example, in csh or
bash setenv NETCDFPAR $NETCDF or export NETCDFPAR=$NETCDF, respectively.
Set the io_form to 13 to use the parallel compressed netcdf option (usually just for the model output or restarts).
This I/O option requires activating the NOCOLONS switch (which is why only model output or restarts are
recommended, since metgrid files have colons imbedded in the file names).

LIST OF MODIFIED FILES:
M Makefile
M Registry/Registry.EM_COMMON
M arch/Config.pl
M arch/md_calls.inc
M arch/postamble
M arch/preamble
M configure
A doc/README.netcdf4par
M external/Makefile
A external/io_netcdfpar/Makefile
A external/io_netcdfpar/diffwrf.F90
A external/io_netcdfpar/ext_ncdpar_get_dom_ti.code
A external/io_netcdfpar/ext_ncdpar_get_var_td.code
A external/io_netcdfpar/ext_ncdpar_get_var_ti.code
A external/io_netcdfpar/ext_ncdpar_put_dom_ti.code
A external/io_netcdfpar/ext_ncdpar_put_var_td.code
A external/io_netcdfpar/ext_ncdpar_put_var_ti.code
A external/io_netcdfpar/field_routines.F90
A external/io_netcdfpar/module_wrfsi_static.F90
A external/io_netcdfpar/transpose.code
A external/io_netcdfpar/wrf_io.F90
M frame/md_calls.m4
M frame/module_io.F
M share/mediation_integrate.F
M share/module_io_domain.F
M share/output_wrf.F
M share/wrf_ext_write_field.F

TESTS CONDUCTED:

Tested on lustre file system (cray) with domain sizes up to about 700x700. Will leave the existing chunking as
as the initial implementation.
Parallel I/O with compression successfully runs on the NCAR cheyenne system, a GPFS file system. The
following were set up for modules during the build and within the job script:

module purge
module load intel
module load ncarcompilers
module load mpt
module load netcdf-mpi
module load ncarenv

The tests on cheyenne worked with both GNU/10.1.0 and Intel/19.1.1
Jenkins testing is OK.
Additional information is now output at the end of the configure step regarding I/O options:

NetCDF version: 4.8.1
Enabled NetCDF-4/HDF-5: yes
NetCDF built with PnetCDF: no
Enabled NetCDF parallel: yes
Using parallel NetCDF via NETCDFPAR option

RELEASE NOTE: Added the ability to write compressed netcdf4 files in parallel via NetCDF 4.7.4 (and later). The performance is slower than pnetcdf, but can be notably faster than regular netcdf4 on parallel file systems. As expected, the compression provides files significantly smaller than pnetcdf generates.

…tcdfpar)

…fpar)

davegill · 2021-09-14T16:45:53Z

@MicroTed
Ted,
This is both great and a huge amount of work! Here are some preliminary questions. This is scheduled for the develop branch, so spring 2022.

Would you please add your affiliation to this PR commit message?
Move the new README file into the top-level "doc" directory.
What are the consequences of NOT having a new enough version of netcdf?
Are there ways to disable the parallel writing?
Is there an accessible "chunk" variable that could be used for tuning?

MicroTed · 2021-09-14T18:01:19Z

@davegill
Thanks!

Would you please add your affiliation to this PR commit message?
-- done

Move the new README file into the top-level "doc" directory.
-- done

What are the consequences of NOT having a new enough version of netcdf?
I did not add any check for netcdf version. I suppose could be done with 'nc-config --version' and some string testing?

Are there ways to disable the parallel writing?
-- Yes, just switch to io_form =2. The regular serial netcdf interface works fine with the parallel-compiled libraries.

Is there an accessible "chunk" variable that could be used for tuning?
-- Hmm, this is a good thought. Like a namelist variable? The testing I've done suggests it is more important to have uniform tile sizes so that they don't overlap chunks. But it would definitely be worth experimenting more with it. I replaced the chunk calculation from the io_netcdf/wrf_io.F90 with a simpler setup that I've used in the COMMAS cloud model, which only does evenly-divisible tiles for domain decomposition.

The NetCDF version 4.7.4 is the earliest that permits the usage of the parallel and compressed options together. modified: configure

modified: configure

modified: Makefile modified: external/io_netcdfpar/Makefile modified: external/io_netcdfpar/wrf_io.F90

…d/WRFV4 into release-v4.3-parallelnc4

new file: external/io_netcdfpar/diffwrf.F90

davegill · 2021-11-17T16:30:42Z

@MicroTed
Ted,

Your mods do not break the standard IO output options. I can get data output with io_form = 2. But, I am having troubles getting this code to work with the new capability. Here is the specific error message:

open_hist_w : error opening wrfout_d01_2021-11-17_06_00_00 for writing. ***

I am using a recent enough version of netcdf:

> nc-config --version
netCDF 4.8.0

The NETCDFPAR env variable points to the correct location:

> echo $NETCDF
/glade/u/apps/ch/opt/netcdf/4.8.0/gnu/11.1.0/
> echo $NETCDFPAR
/glade/u/apps/ch/opt/netcdf/4.8.0/gnu/11.1.0/

The namelist is set up to output the new format:

 &time_control
 nocolons                            = .true.
 io_form_history                     = 13
 /

The code is built for MPI. I have tried various processor counts, from 1 through 36. The domain is relatively small, so there should not be any troubles in that regard.

Any ideas or suggestions?

MicroTed · 2021-11-17T17:23:45Z

@davegill
Such a helpful error message, isn't it? I have only tested with netcdf 4.7.4 and Intel fortran. I expect you have it set up correctly, and it should work on any file system (serial disk just has no speedup). Nothing comes to mind, though I suppose it is possible that there is a bug affecting gfortran but not ifort? I could give 4.8.0 and gfortran a try myself, but I'm a bit strapped for time at the moment -- I'll see if I can test that.

davegill · 2021-11-17T18:13:38Z

@MicroTed
Ted,
I have tried with Intel/21 and GNU/11, both with netcdf/4.8.0

MicroTed · 2021-11-18T02:32:51Z

The netcdf 4.8.0 release has a possibly useful new test:
[Enhancement] Added new test for using compression with parallel I/O: nc_test4/tst_h_par_compress.c. See [https://github.com//pull/1784].

P.S. I compiled 4.8.0 (on mac) and that also worked with the PR code, so it doesn't seem to be a problem with 4.8.0 itself.

modified: share/mediation_integrate.F

… netcdf output.

MicroTed · 2021-12-23T02:43:00Z

@davegill
I found the problem with the metadata, which was that I missed porting the update to frame/md_calls.m4. I pushed those changes (also to the backup arch/md_calls.inc).

davegill · 2022-01-05T19:19:07Z

@MicroTed @weiwangncar @kkeene44
Ted,
Thanks again for this contribution!

Wei and Kelly,
This is ready for review. I have worked with Ted to iron out the kinks on my end, and have added in the required diffwrf utility. It is a niche capability, but it is fully functional.

weiwangncar · 2022-01-05T20:17:28Z

@davegill Have you retested code with this latest change?

davegill · 2022-01-05T20:20:41Z

@weiwangncar

Have you retested code with this latest change?

Wei,
This is tested, it works, and it is ready to go. For quite a while, the only trouble was (unbeknownst to me) getting the environment INSIDE of the qsub'ed job to be identical to the environment used during the build.

TYPE: bug fix KEYWORDS: netcdfpar, Error SOURCE: internal DESCRIPTION OF CHANGES: Problem: With PR wrf-model#1552 "Parallel netcdf-4 IO option" (SHA1 3cd4713), when then code was built without the new parallel NetCDF4 compression, the build log had an `Error`. The problem was related to constructing the object files in the io_netcdfpar directory. When the option is not selected at compile time, then we do not care about errors in the directory that will never be used. Solution: If the NETCDFPAR option is not selected at compile time, then do SKIP going into the io_netcdfpar directory all together. LIST OF MODIFIED FILES: m Makefile m arch/Config.pl m arch/configure.defaults TESTS CONDUCTED: 1. Without the NETCDFPAR parameter set, the build for the io_netcdfpar directory is bypassed: ``` cd ../io_netcdfpar ; \ echo SKIPPING make -i -r NETCDFPARPATH="/glade/u/apps/ch/opt/netcdf/4.7.3/gnu/9.1.0" \ cd ../io_netcdfpar ; \ echo SKIPPING make -i -r NETCDFPARPATH="/glade/u/apps/ch/opt/netcdf/4.7.3/gnu/9.1.0" \ ``` 2. When the NETCDFPAR env variable is set, the build includes the io_netcdfpar directory: cd ../io_netcdfpar ; \ make -i -r NETCDFPARPATH="/glade/u/apps/ch/opt/netcdf/4.8.0/gnu/9.1.0" \ cd ../io_netcdfpar ; \ make -i -r NETCDFPARPATH="/glade/u/apps/ch/opt/netcdf/4.8.0/gnu/9.1.0" \ ``` 3. Jenkins tests are all PASS. RELEASE NOTE: Include a stand-alone message suitable for the inclusion in the minor and annual releases. A publication citation is appropriate.

TYPE: bug fix KEYWORDS: netcdfpar, Error SOURCE: internal DESCRIPTION OF CHANGES: IMPORTANT: Without these mods, every commit since the parallel netcdf4 IO mods will fail the DA build test in the regression test. For example, at least these commits: ``` fed10f4 Adding the WRF-Solar EPS model (#1547) 0bda5e0 Fix 4dvar build failure after commit 8b5bfe5 (#1652) 8b5bfe5 Thompson AA enhancements: BC aerosol, biomass burning emissions, and … (#1616) 9dc68ca After testing with UFS/GFS/FV-3, some tuning knob changes to Thompson-MP and icloud3 (cloud fraction) scheme (#1626) 96fd889 Update HONO, TERP, and CO2 emissions (#1644) 64fb190 SFCLAY=1, add shallow water roughness calculation (#1543) 609c2fc New module firebrand_spotting for WRF-Fire (#1540) 75bfe6d MYNN PBL clouds in photolysis option 4 (TUV) (#1622) f8c4b13 Fix runtime error when using sf_surface_mosaic = 1 with use_wudapt_lcz = 0 (#1638) b511c70 Run-time option for climate GHG for radiation (#1625) 8194c66 Bug fix for configuration option INTEL:HSW/BDW (#1645) 16c9287 bug fixes for radar_rf_opt=2 (#1642) a82ce24 Sync with NoahMP Github version with all NoahMP updates since v4.3 (#1641) 7b642cc Bug fix for TAMDAR T VarBC (#1632) 92fd706 fix WRFDA build for Parallel netcdf-4 IO (#1634) ``` Problem: With PR #1552 "Parallel netcdf-4 IO option" (SHA1 3cd4713), when then code was built without the new parallel NetCDF4 compression, the build log had an `Error`. ``` > grep Error compile.log Fatal Error: Cannot open module file ‘wrf_data_ncpar.mod’ for reading at (1): No such file or directory make[2]: [diffwrf] Error 1 (ignored) make[2]: [diffwrf] Error 1 (ignored) wrf_io.f:117: Error: Can't open included file 'mpif.h' make[2]: [wrf_io.o] Error 1 (ignored) Fatal Error: Cannot open module file ‘wrf_data_ncpar.mod’ for reading at (1): No such file or directory make[2]: [field_routines.o] Error 1 (ignored) make[2]: [libwrfio_nfpar.a] Error 127 (ignored) make[2]: [libwrfio_nfpar.a] Error 1 (ignored) ``` The problem was related to constructing the object files in the io_netcdfpar directory. When the option is not selected at compile time, then we do not care about errors in the directory that will never be used. Solution: If the NETCDFPAR option is not selected at compile time, then SKIP going into the io_netcdfpar directory all together. LIST OF MODIFIED FILES: m Makefile m arch/Config.pl m arch/configure.defaults m configure TESTS CONDUCTED: 1. Without the NETCDFPAR parameter set, the build for the io_netcdfpar directory is bypassed: ``` cd ../io_netcdfpar ; \ echo SKIPPING make -i -r NETCDFPARPATH="/glade/u/apps/ch/opt/netcdf/4.7.3/gnu/9.1.0" \ cd ../io_netcdfpar ; \ echo SKIPPING make -i -r NETCDFPARPATH="/glade/u/apps/ch/opt/netcdf/4.7.3/gnu/9.1.0" \ ``` 2. When the NETCDFPAR env variable is set, the build includes the io_netcdfpar directory: cd ../io_netcdfpar ; \ make -i -r NETCDFPARPATH="/glade/u/apps/ch/opt/netcdf/4.8.0/gnu/9.1.0" \ cd ../io_netcdfpar ; \ make -i -r NETCDFPARPATH="/glade/u/apps/ch/opt/netcdf/4.8.0/gnu/9.1.0" \ ``` 3. Jenkins tests are all PASS.

TYPE: bug fix KEYWORDS: netcdfpar, Error SOURCE: internal DESCRIPTION OF CHANGES: Problem: With PR wrf-model#1552 "Parallel netcdf-4 IO option" (SHA1 3cd4713), when then code was built without the new parallel NetCDF4 compression, the build log had an `Error`. The problem was related to constructing the object files in the io_netcdfpar directory. When the option is not selected at compile time, then we do not care about errors in the directory that will never be used. Solution: If the NETCDFPAR option is not selected at compile time, then do SKIP going into the io_netcdfpar directory all together. LIST OF MODIFIED FILES: m Makefile m arch/Config.pl m arch/configure.defaults TESTS CONDUCTED: 1. Without the NETCDFPAR parameter set, the build for the io_netcdfpar directory is bypassed: ``` cd ../io_netcdfpar ; \ echo SKIPPING make -i -r NETCDFPARPATH="/glade/u/apps/ch/opt/netcdf/4.7.3/gnu/9.1.0" \ cd ../io_netcdfpar ; \ echo SKIPPING make -i -r NETCDFPARPATH="/glade/u/apps/ch/opt/netcdf/4.7.3/gnu/9.1.0" \ ``` 2. When the NETCDFPAR env variable is set, the build includes the io_netcdfpar directory: cd ../io_netcdfpar ; \ make -i -r NETCDFPARPATH="/glade/u/apps/ch/opt/netcdf/4.8.0/gnu/9.1.0" \ cd ../io_netcdfpar ; \ make -i -r NETCDFPARPATH="/glade/u/apps/ch/opt/netcdf/4.8.0/gnu/9.1.0" \ ``` 3. Jenkins tests are all PASS. RELEASE NOTE: Include a stand-alone message suitable for the inclusion in the minor and annual releases. A publication citation is appropriate.

TYPE: new feature KEYWORDS: parallel netcdf4 SOURCE: Ted Mansell (NOAA/National Severe Storms Lab) DESCRIPTION OF CHANGES: This PR adds an I/O option to use the new netcdf capability in 4.7.4 (and later versions) to write both in parallel and with variable compression for the same file. Versions of pnetcdf permitted parallel I/O, but did not permit standard HDF5 compression. A new I/O library directory, `external/io_netcdfpar`, has been added that is modeled on the similar `external/io_netcdf` directory. Users should see the file doc/README.netcdf4par for details on configuring and running. In a nutshell, steps to use this feature: 1. This capability to use HDF5 compression and to have parallel I/O (similar to pnetcdf) requires NetCDF v4.7.4 or later. 2. The NetCDF library must be built with MPI. 3. Once the NetCDF library / module is chosen and the NETCDF env variable is correctly pointing to the right location, then prior to the `configure` step, the user sets a new environment variable. For example, in csh or bash `setenv NETCDFPAR $NETCDF` or `export NETCDFPAR=$NETCDF`, respectively. 4. Set the io_form to 13 to use the parallel compressed netcdf option (usually just for the model output or restarts). 5. This I/O option requires activating the NOCOLONS switch (which is why only model output or restarts are recommended, since metgrid files have colons imbedded in the file names). LIST OF MODIFIED FILES: M Makefile M Registry/Registry.EM_COMMON M arch/Config.pl M arch/md_calls.inc M arch/postamble M arch/preamble M configure A doc/README.netcdf4par M external/Makefile A external/io_netcdfpar/Makefile A external/io_netcdfpar/diffwrf.F90 A external/io_netcdfpar/ext_ncdpar_get_dom_ti.code A external/io_netcdfpar/ext_ncdpar_get_var_td.code A external/io_netcdfpar/ext_ncdpar_get_var_ti.code A external/io_netcdfpar/ext_ncdpar_put_dom_ti.code A external/io_netcdfpar/ext_ncdpar_put_var_td.code A external/io_netcdfpar/ext_ncdpar_put_var_ti.code A external/io_netcdfpar/field_routines.F90 A external/io_netcdfpar/module_wrfsi_static.F90 A external/io_netcdfpar/transpose.code A external/io_netcdfpar/wrf_io.F90 M frame/md_calls.m4 M frame/module_io.F M share/mediation_integrate.F M share/module_io_domain.F M share/output_wrf.F M share/wrf_ext_write_field.F TESTS CONDUCTED: 1. Tested on lustre file system (cray) with domain sizes up to about 700x700. Will leave the existing chunking as as the initial implementation. 2. Parallel I/O with compression successfully runs on the NCAR cheyenne system, a GPFS file system. The following were set up for modules during the build and within the job script: ``` module purge module load intel module load ncarcompilers module load mpt module load netcdf-mpi module load ncarenv ``` 3. The tests on cheyenne worked with both GNU/10.1.0 and Intel/19.1.1 4. Jenkins testing is OK. 5. Additional information is now output at the end of the `configure` step regarding I/O options: ``` NetCDF version: 4.8.1 Enabled NetCDF-4/HDF-5: yes NetCDF built with PnetCDF: no Enabled NetCDF parallel: yes Using parallel NetCDF via NETCDFPAR option ``` RELEASE NOTE: Added the ability to write compressed NetCDF4 files in parallel via NetCDF 4.7.4 (and later). The performance is slower than pnetcdf, but can be notably faster than regular NetCDF4 on parallel file systems. As expected, the compression provides files significantly smaller than pnetcdf generates.

TYPE: bug fix KEYWORDS: netcdfpar, Error SOURCE: internal DESCRIPTION OF CHANGES: IMPORTANT: Without these mods, every commit since the parallel netcdf4 IO mods will fail the DA build test in the regression test. For example, at least these commits: ``` fed10f4 Adding the WRF-Solar EPS model (wrf-model#1547) 0bda5e0 Fix 4dvar build failure after commit 8b5bfe5 (wrf-model#1652) 8b5bfe5 Thompson AA enhancements: BC aerosol, biomass burning emissions, and … (wrf-model#1616) 9dc68ca After testing with UFS/GFS/FV-3, some tuning knob changes to Thompson-MP and icloud3 (cloud fraction) scheme (wrf-model#1626) 96fd889 Update HONO, TERP, and CO2 emissions (wrf-model#1644) 64fb190 SFCLAY=1, add shallow water roughness calculation (wrf-model#1543) 609c2fc New module firebrand_spotting for WRF-Fire (wrf-model#1540) 75bfe6d MYNN PBL clouds in photolysis option 4 (TUV) (wrf-model#1622) f8c4b13 Fix runtime error when using sf_surface_mosaic = 1 with use_wudapt_lcz = 0 (wrf-model#1638) b511c70 Run-time option for climate GHG for radiation (wrf-model#1625) 8194c66 Bug fix for configuration option INTEL:HSW/BDW (wrf-model#1645) 16c9287 bug fixes for radar_rf_opt=2 (wrf-model#1642) a82ce24 Sync with NoahMP Github version with all NoahMP updates since v4.3 (wrf-model#1641) 7b642cc Bug fix for TAMDAR T VarBC (wrf-model#1632) 92fd706 fix WRFDA build for Parallel netcdf-4 IO (wrf-model#1634) ``` Problem: With PR wrf-model#1552 "Parallel netcdf-4 IO option" (SHA1 3cd4713), when then code was built without the new parallel NetCDF4 compression, the build log had an `Error`. ``` > grep Error compile.log Fatal Error: Cannot open module file ‘wrf_data_ncpar.mod’ for reading at (1): No such file or directory make[2]: [diffwrf] Error 1 (ignored) make[2]: [diffwrf] Error 1 (ignored) wrf_io.f:117: Error: Can't open included file 'mpif.h' make[2]: [wrf_io.o] Error 1 (ignored) Fatal Error: Cannot open module file ‘wrf_data_ncpar.mod’ for reading at (1): No such file or directory make[2]: [field_routines.o] Error 1 (ignored) make[2]: [libwrfio_nfpar.a] Error 127 (ignored) make[2]: [libwrfio_nfpar.a] Error 1 (ignored) ``` The problem was related to constructing the object files in the io_netcdfpar directory. When the option is not selected at compile time, then we do not care about errors in the directory that will never be used. Solution: If the NETCDFPAR option is not selected at compile time, then SKIP going into the io_netcdfpar directory all together. LIST OF MODIFIED FILES: m Makefile m arch/Config.pl m arch/configure.defaults m configure TESTS CONDUCTED: 1. Without the NETCDFPAR parameter set, the build for the io_netcdfpar directory is bypassed: ``` cd ../io_netcdfpar ; \ echo SKIPPING make -i -r NETCDFPARPATH="/glade/u/apps/ch/opt/netcdf/4.7.3/gnu/9.1.0" \ cd ../io_netcdfpar ; \ echo SKIPPING make -i -r NETCDFPARPATH="/glade/u/apps/ch/opt/netcdf/4.7.3/gnu/9.1.0" \ ``` 2. When the NETCDFPAR env variable is set, the build includes the io_netcdfpar directory: cd ../io_netcdfpar ; \ make -i -r NETCDFPARPATH="/glade/u/apps/ch/opt/netcdf/4.8.0/gnu/9.1.0" \ cd ../io_netcdfpar ; \ make -i -r NETCDFPARPATH="/glade/u/apps/ch/opt/netcdf/4.8.0/gnu/9.1.0" \ ``` 3. Jenkins tests are all PASS.

MicroTed added 3 commits September 8, 2021 19:27

Updated files for parallel netcdf-4 with internal compression (io_ne…

78b2a70

…tcdfpar)

Added files for parallel netcdf-4 with compression (external/io_netcd…

a3666d6

…fpar)

Draft README for compiling/using parallel netcdf-4

8d30f4d

MicroTed requested review from a team as code owners September 9, 2021 00:55

Updated README

634bc75

davegill changed the base branch from release-v4.3 to develop September 14, 2021 16:46

davegill added Enhancement Develop Branch Infrastructure: OK labels Sep 14, 2021

Moved readme file to doc folder

a2e85bb

MicroTed requested a review from a team as a code owner September 14, 2021 17:16

davegill added 4 commits September 16, 2021 12:24

parnetcdf4 trigger reggie without NMM

346ef78

parnetcdf trigger wrf-model#2

4bb5d9c

par2 test wrf-model#3 reggie 13:47

94fb542

Fix logic so that 4.7.4 is minimum for NETCDFPAR usage

f177aa8

The NetCDF version 4.7.4 is the earliest that permits the usage of the parallel and compressed options together. modified: configure

davegill added the Feature label Nov 3, 2021

davegill added 3 commits November 3, 2021 14:59

Feature testing

a79a72c

Hmm, once more

35cb651

Try a push from command line with Feature label

7065a83

modified: configure

davegill removed the Feature label Nov 7, 2021

davegill added 3 commits November 10, 2021 00:54

Fix diffwrf with NETCDFPAR

09a3fe2

modified: Makefile modified: external/io_netcdfpar/Makefile modified: external/io_netcdfpar/wrf_io.F90

Merge branch 'release-v4.3-parallelnc4' of https://github.com/MicroTe…

9d72426

…d/WRFV4 into release-v4.3-parallelnc4

oops, add diffwrf.F90 for netcdfpar, oops

618fd5e

new file: external/io_netcdfpar/diffwrf.F90

Error codes from NETCDF are larger than I3

9711a5a

modified: share/mediation_integrate.F

davegill added Feature and removed Enhancement Develop Branch Infrastructure: OK labels Dec 9, 2021

netcdf4 22:43

2796176

davegill added Enhancement Develop Branch Infrastructure: OK and removed Feature labels Dec 9, 2021

Added missing changes to md_calls.m4, which fixes missing metadata in…

3cd4713

… netcdf output.

davegill requested review from davegill, kkeene44 and weiwangncar and removed request for a team January 5, 2022 19:19

davegill approved these changes Jan 5, 2022

View reviewed changes

weiwangncar approved these changes Jan 5, 2022

View reviewed changes

davegill merged commit e018624 into wrf-model:develop Jan 5, 2022

davegill mentioned this pull request Jan 22, 2022

BF: NETCDFPAR "Error" in build log when option not used #1651

Merged

stephenmsachs mentioned this pull request Jan 27, 2023

wrf: add v4.2.2 spack/spack#35202

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Parallel netcdf-4 IO option#1552

Parallel netcdf-4 IO option#1552
davegill merged 18 commits intowrf-model:developfrom
MicroTed:release-v4.3-parallelnc4

MicroTed commented Sep 9, 2021 •

edited by davegill

Loading

Uh oh!

davegill commented Sep 14, 2021 •

edited

Loading

Uh oh!

MicroTed commented Sep 14, 2021

Uh oh!

davegill commented Nov 17, 2021

Uh oh!

MicroTed commented Nov 17, 2021

Uh oh!

davegill commented Nov 17, 2021

Uh oh!

MicroTed commented Nov 18, 2021 •

edited

Loading

Uh oh!

MicroTed commented Dec 23, 2021

Uh oh!

davegill commented Jan 5, 2022

Uh oh!

weiwangncar commented Jan 5, 2022

Uh oh!

davegill commented Jan 5, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

MicroTed commented Sep 9, 2021 • edited by davegill Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

davegill commented Sep 14, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MicroTed commented Sep 14, 2021

Uh oh!

davegill commented Nov 17, 2021

Uh oh!

MicroTed commented Nov 17, 2021

Uh oh!

davegill commented Nov 17, 2021

Uh oh!

MicroTed commented Nov 18, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MicroTed commented Dec 23, 2021

Uh oh!

davegill commented Jan 5, 2022

Uh oh!

weiwangncar commented Jan 5, 2022

Uh oh!

davegill commented Jan 5, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

MicroTed commented Sep 9, 2021 •

edited by davegill

Loading

davegill commented Sep 14, 2021 •

edited

Loading

MicroTed commented Nov 18, 2021 •

edited

Loading