Merge develop to master #302
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
There is a subtask for remapping observations and one for making each plot (that is, one per season and comparison grid).
As with the ocean, there is subtask for remapping observations (which actually does very little since the observations are already on the comparison grid) and one subtask for each plot (meaning each season and, for concentration, for each set of observations).
SST, SSS and MLD can be plotted on this grid by adding 'antarctic' to the comparisonGrids list. A plotting routine for data on a ploar grid has also been added, including support for both index and norm color maps (the latter will be used for melt rates and Antarctic temperatures at various depths).
This merge creates a generic subtask for remapping observations to the comparison grid that can be used by both ocean and sea-ice tasks. Tasks for ocean and sea-ice climatology maps are broken out into their own files, each of which defines the task and a subtask for reading in the observational data set and defining the source mesh/grid for the observations. In each case, the main task now simply spawns subtasks for remapping model results and observations and for plotting the results and is otherwise empty.
There was a conflict between the name of the module and the name of the class that was confusing sphinx.
Fix imports of config parser Remove test for obs. climatology file name, which has moved to the remap obs. climatology subtask.
Split climatology maps into more subtasks For both ocean and sea-ice climatology maps, there is a subtask for remapping observations and one for making each plot (that is, one per season, comparison grid and set of observations). SST, SSS and MLD can be plotted on an Antarctic stereographic grid by adding 'antarctic' to the comparisonGrids list. A plotting routine for data on a polar grid has also been added, including support for both index and norm color maps (the latter will be used for melt rates and Antarctic temperatures at various depths). The climatology map plotting subtask under each component has been split out into its own file, the remapping subtasks are under mpas_analysis.shared.climatology and each climatology map task has been split into its own file (e.g. climatology_map_mld.py) to make it easier to figure out what is going on.
This task plots climatologies of Antarctic melt rates against observations from Rignot et al. (2013). To support masking the melt rate field only where land-ice is present and renormalizing the field to change to units of m/yr, a capability to customize climatologies during the remapping process. A new directory for the melt observations has been added, along with a new section in the config file for this task.
For runs without ice-shelf cavities, remove the climatologyMapAntarcticMelt task from the generate list Edit the description of the generate option to refer to ./run_mpas_analysis --list so we don't have to keep updating the config files when new tasks and tags are added.
The dimensions can't simply be read from the file because they have been renamed in the data set and this confuses the remapping routine.
For some versions of matplotlib, this seems to be necessary for certain plots (notably the melt-rate plots).
Otherwise the plot with pluses can become indistinguishable from a thick line.
Add task for plotting climatology maps for Antarctic melt rates xylar commented a day ago Observations are from Rignot et al. (2013). To support masking the melt rate field only where land-ice is present and renormalizing the field to change to units of m/yr, a capability to customize climatologies during the remapping process. A new directory for the melt observations has been added, along with a new section in the config file for this task. The new task has been added to the auto-documentation. 'no_climatologyMapAntarcticMelt' has been added to generate lists for all example config files without land-ice cavities. This way, users aren't concerned when this task fails during setup_and_check(). Small bug fixes have been included for: * masking plot variables as numpy masked arrays in the ocean plotting routines * creating a grid descriptor for MLD observations that uses dimensions `lat` and `lon` rather than `iLAT` and `iLON`.
The name of the mask file is constructed automatically from the mesh name.
Remove --purge option from job scripts
Change to mask directory instead of individual mask files The name of the MOC mask file is constructed automatically from the mesh name. This approach is needed in anticipation of having other region masks (notably masks for individual ice shelves, see #237) Example config files have been updated accordingly.
This merge makes the range of files printed in the log ignore the first year if it is not in the requested range, even though it gets added to allow anomaly computations.
Fix first file for time series This merge makes the range of files printed in the log ignore the first year if it is not in the requested range, even though it gets added to allow anomaly computations.
...in preparation for adding a panel of a reference run. A function for helping to determine the extent of the y axis for spectra has been moved from plotting into this task and has also been cleaned up/simplified
The prefix about a UserWarning with a useless stacktrace in front of warnings produced by warnings.warn are just confusing.
Clean up the nino34 index task This clean up is needed to generalize the task in preparation for supporting comparison with a reference run (which will replace one of the panels). A new task has been added to extract time series needed by climate indices (which may have different time bounds than time series). Support for separate time bounds was eliminated, perhaps accidentally, in #271. Nino34 spectra are now packed into dictionaries for easier iteration and panels are plotted in loops. The function for determining the maximum value of all spectra (to determine the bounds of the y axes of these plots) has been cleaned up so it no longer relies on a plot already having been performed). All warnings previously produced with warnings.warn have been switched to just using print('Warning:..') because this produces much more intuitive output without a (nearly always useless) stack trace.
Make clear which items are tasks, rather than tags. Add printing of the component to hopefully indicate that this can also be used as a flag.
Add -4 flag to force NetCDF4 output format from NCO This prevents certain issues with very large NetCDF3 files (e.g. from high resolution MPAS-O output). Without this fix, an error occurs when running analysis on large MPAS files: netCDF3 NETCDF_64BIT_OFFSET format limits fixed variables to sizes smaller than 2^32 B = 4 GiB ~ 4.2 GB, and record variables to that size per record.
Also, a bit of other clean up in the config file -- removing unused arguments and fixing mesh names in a comment.
These tasks just make sure the climatology and remapped climatology files exist in the reference run. The task name of MpasClimatolgyTask can optionally be set, so that the reference run's task doesn't have the same name as the equivalent for the main run Similarly, the remapping subtask needs a way to set its subtask.
This task just makes sure the expected file is there and has the expected variables.
The routines are plot_polar_comparison, plot_global_comparison and plot_polar_projection_comparison. This chanrge has been made because the second panel may be used to plot a reference run instead of observations, depending on the particular analysis configuration. Also, each routine plots only 1 panel instead of 3 if there is no refArray. This is useful for plots with no observations or reference run to compare against.
The task now takes an optional task that remaps a reference run instead of observations. Support has also been added for having neither a reference run nor observations (e.g. for plotting climatologies where no observation are available and no reference run has been specified).
The task now takes an optional task that remaps a reference run instead of observations. Support has also been added for having neither a reference run nor observations (e.g. for plotting climatologies where no observation are available and no reference run has been specified).
All climatology and time series tasks except MOC now can be plotted against a reference run. Support for plotting the MOC against a reference run is not likely to be supported.
This also requires writing out the aggregated melt time-series data set, so that this data set can simply be read in from the reference run. Because it is easy to include, this merge also loads the time series data for the main run if the data has already been cached.
Also add a few other items missed by recent commits
Instead, use the ref config. file toa construct file names without the need for subtasks. This removes any need for the reference simulation data to exist; only the analysis results are required. funcitons for constructing unmasked, masked and remapped MPAS file names have been added to shared.climatology. These functions are now used in the associated climatology tasks/subtasks. All tasks that support plotting reference data have been updated with this change. The documentation has also been updated.
This is needed so it can be read back in from the reference run without needing the mesh from the reference run.
The MHT latitude bins, depths and climatology are stored in a file so these can be read back in for a reference run.
It is commented out but it (hopefully) makes clear how one would point to a reference run.
This allows comparison between runs with different vertical coordinates
Also, print message that reference analysis should already have been run (if a ref config file is specified).
Update list of authors Some recent authors were not included by mistake. Authors are ordered by their contributions to the code (both additions and PRs reviewed). Also, simplifies the instructions for building the docs and updates the version number in the docs.
Support comparison with a reference run This merge adds the capability to compare the output of a simulation against a reference simulation. In the case of climatologies, the comparison with the reference run is performed instead of the comparison with observations (or state estimates, reanalyses, etc.). In the case of time series, an additional plot is added for the reference run. Notes: * Legend information has been moved from the title to the legend in all plots, since this is much easier to handle in the case of plots with anywhere from 1 to 4 time series * Comparison with a reference run is not supported for the MOC (though this could conceivably be added in a future PR) * It isn't obvious how to do a meaningful comparison of Hovmoller (trend vs. depth) plots with a reference run, so this isn't currently supported The reference run is specified through the [runs]/referenceRunConfigFile option, which points to a config file describing analysis that must have already been performed on the reference run. This saves computing the reference climatologies and time series again and again if the same reference run is used several times. (It also seems to fix some NetCDF/HDF5 errors I encountered when trying to compare several runs at once to the same reference run.) Additionally, the reference config file can specify a range of years for the climatology and/or time series that differs from the main run. The startYear and endYear for climatologies, time series and indices in the reference config file must be the actual range that was used to compute them. Otherwise, the results may not be found correctly or the results may not plot correctly. One way to make sure this is the case is to use the config file saved to the logs directory in when the reference analysis is performed.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Merge PRs #282, #285, #287, #289, #290, #291, #293, #295, #288, #296, #237, #297, #294, #300 and #292