EPPT-2510: Adding calculation of saturation vapour pressure derivative in air #2165
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…s so it correctly finds the function
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## EPPT_2072_contrails_design #2165 +/- ##
=============================================================
Coverage ? 98.48%
=============================================================
Files ? 141
Lines ? 13838
Branches ? 0
=============================================================
Hits ? 13629
Misses ? 209
Partials ? 0 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
mspelman07
approved these changes
Jul 28, 2025
Contributor
mspelman07
left a comment
mspelman07
left a comment
There was a problem hiding this comment.
Happy to approve this. The tests all pass and the code seems to match the intended equations.
| @@ -0,0 +1,50 @@ | |||
| # (C) Crown Copyright, Met Office. All rights reserved. | |||
Contributor
There was a problem hiding this comment.
Typically when we add new tests we do it in pytest (the old tests all use iris test). This doesn't actually afffect anything functionally so I don't think it's worth changing anything
ryan-cocking-mo
approved these changes
Jul 29, 2025
Contributor
ryan-cocking-mo
left a comment
ryan-cocking-mo
left a comment
There was a problem hiding this comment.
Tests all pass and the code matches the equations as written.
mo-philrelton
merged commit 91024ee
into
metoppv:EPPT_2072_contrails_design
10 checks passed
2 tasks
mo-philrelton
pushed a commit
to mo-philrelton/improver
that referenced
this pull request
Sep 5, 2025
…e in air (metoppv#2165) * Code changes to add the SVP derivative corrected for in air * Adding a missing temperature term to the correction calculation * Updating expected values in unit tests and changing one of the imports so it correctly finds the function * Updating one of the ecpected values within the unit tests
mo-philrelton
added a commit
that referenced
this pull request
Sep 8, 2025
* Fill value (#2129) * update save function to include fill value * update docstring * update docstring * Update tests for fill value * 2154 BUG: Convective cloud base and top can be unphysical (#2155) * Limits cloud base pressure and temperature to surface values if super-saturated * Updates checksums for revised KGO * Fix recalibration docstring (#2166) * Use np.interp instead of scipy interp1d * Revert "Use np.interp instead of scipy interp1d" This reverts commit 3ea6cd6. * fix docstring --------- Co-authored-by: Belinda Trotta <[email protected]> * Environment upgrade feature branch (#2167) * First cut changes in preparation for environment upgrade (#2124) * Avoid cubelists of cubelists. * Replace np.product with np.prod. * Replace np.int. * Replace np.NAN with np.nan. * Replace np.NAN with np.nan. * Replace np.NaN with np.nan. * Replace np.product with np.prod. * Replace assertRaisesRegexp with assertRaisesRegex and use collections.abc.Callable instead of collections.Callable. * Simplify test in test_flatten.py to avoid cubelist within a cubelist. * Replace Cube(None) with an alternative. * Unpin environments for testing. * Remove pygam version. * Update improver_a.yml * Remove non-essential dependencies from yml files and add pins. * Minor edits following review comments. * Modify docstring to better reflect the inputs provided. * Updates to cube_combiner for new environment (#2128) * Updates to cube_combiner for new environment * Update checksums * Mobt913: Environment upgrade - Metadata (#2137) * Fix failing tests following environment change. * Handle bug where num2date is returning cftime datetime instead of python datetime where it shouldn't * Mobt915: environment upgrade - Orographic enhancement (#2138) * Cast variable to float32 to resolve environment upgrade error * Fix pre-commit reformatting * Environment upgrade: expected value (#2139) * Modify expected scalar values in unit tests to have specific float32 type to pass more rigorous numpy assert_allclose test. * Update checksums * Mobt 914 env update nowcasting (#2142) * Ensures precision of output data are float32 * Reduces precision requirement of unit tests from 7 decimal places to 6. * Fixes unit tests that were passing the wrong arguments to the methods they were testing * Developer_tools: Updates metadata interpreter to print dict-like strings from CubeAttrsDict objects (#2134) * Updates metadata interpreter to print dict-like strings from CubeAttrsDict objects * Improves imports * Simplifies conversion of cube_attrs to dict * Environment upgrade: spot data (#2141) * Change expected results in neighbour selection tied cases due to slight variation in return from coordinate transform. * Change from using numpy float type to native float type when converting type of user provided percentiles. * Fix up acceptance test which collects warnings. Update checksums. * Mobt 916 environment upgrade regrid (#2144) * Resolve failings tests following environment upgrade * Fix pre_commit requirements * Remove print statment and remove lots of unnecessarily added trailing .0s. --------- Co-authored-by: benjamin.ayliffe <[email protected]> * Upgrade utilities unit tests (#2140) * Remove test for non-Cube inputs to a CubeList, as this is no longer possible. * Unpack list for use when slicing in cube_extraction. * Edits to test_temporal.py. * Correction to datatypes within temporal_interpolation.py. * Corrections to gradient_between_vertical_levels * GAM corrections copied from #2126 for completeness. * Edits, so that load unit tests pass, although we're no longer testing the lazy loading successfully, so this may need reconsideration. * Alter ordering of bounds in mathematical_operations.py due to underlying change in iris. * Retain intended indexing behaviour within neighbourhood_tools.py by converting list to tuple. * Changes to allow more tolerance for solar interpolation tests, where data type differences can impact precision. * Update load unit tests to override iris setting that prevents lazy loading for small files. * Revert changes to solar.py, which are not required, following allowing a greater tolerance in the test_solar_interpolation unit test. * Minor edit to test_load.py * Minor edit to return a python datetime from the iris_time_to_datetime function, rather than a cftime datetime object. * Simplification to use to_real_datetime method available in iris. * Mobt906 Calibration unit tests upgrades for new environment (#2127) * Fix tests in dz_rescaling. Added re.escape() call to regex pattern match. * Update IMPROVER choose() function as numpy.lib.index_tricks.ndindex() moved to numpy.ndindex(). * Fix faulty plugin call in improver_tests/calibration/test_init.py. * Fix tests in reliability_calibration. Added re.escape() call to regex pattern match. * Fix tests in calibration/utilities. Added re.escape() call to regex pattern match. * Modify rainforests_calibration/conftest.py so that both treelite and treelite_runtime are checked for before attempting to run relevant tests, rather than only treelite. * Change expected results for failing ensemble_calibration/test_EstimateCoefficientsForEnsembleCalibration.py tests. * Add treelite_runtime module dependency to environments containing treelite as a dependency. * Revert changes related to rainforest calibration unit tests and treelite_runtime. More work is required to properly handle upgrading from treelite 3.x to 4.y. * Change expected results for 2 tests in test_CalibratedForecastDistributionParameters.py. * Revert changes to tests which were failing due to Regex pattern matching issues with numpy types being displayed. Instead, map these objects to base Python types, e.g str instead of np.str. * Environment upgrade: generate ancillaries (#2143) * Make constant float64 type where it is used to preserve original results. * Changes to orographic smoothing coefficient generation to avoid type escalation to float64. * Formatting update. * enforce data type for cloud condensation level plugins (#2146) * Acceptance test batch 2 (#2148) * Enforce float32. * Enforce input to cube.collapsed is the same type as the output. * Get dtype without using .data. * Modification to enforce the datatype within the mode_aggregator method and add an assertion to the unit tests to show that the output dtype matches the input dtype. * Acceptance test batch 5 (#2149) * Minor edit to textural.py * Move setting of data type on the threshold values into the threshold plugin. * Enforce data type (#2147) * Update environment for rainforests calibration (#2136) This patch provides changes to the apply_rainforests_calibration plugin in order to support the IMPROVER repo environment update (see Github issue #150). Specifically, this patch sets lower bound versions for the treelite and lightgbm packages to support an up-to-date Python environment. Corresponding code changes for these version upgrades are included. Furthermore, I have added a few minor quality changes to the rainforests code - see below list. This changeset includes: * Set lower bound for treelite and lightgbm package versions * Add tl2cgen package (needed for treelite v4.0.0 and up) * Changes to rainforests calibration code to support updated package versions * Update docs to reflect new package versions * Quality of life changes to rainforests calibration code: * Ensure correct type annotations are used * Use custom exceptions where feasible * Use StrEnum for model name to constrain possible argument values * Improved error handling * Additional clarity in docstrings * Changes to rainforests calibration unit tests: * Update unit tests to support the above changes * Add additional rainforests calibration unit tests * Move some code that supports unit tests, to avoid duplication * Update rainforests calibration acceptance tests and checksums Relates to Github issue #150 * Resolve calendar issues in kgos, checksums, and a depracated warning (#2151) * Acceptance test batch 11 (#2150) * Resolve calendar and float dtype issues following environment upgrade * Repeat for weighted_blending * Mobt 943 acceptance tests batch 9 (#2152) * fixes mobt_943_acceptance_tests_batch_9 following environment changes - Update calendars on KGOs - Fix returning of float32 instead of in8 for categorical inputs - Remove surprising use of iris.cube.CubeList() in acceptance test, causing test failures * Pre-commit formatting fixes * Simplify typecasting from previous commit * Remove change of checksums so left until end of all acceptance testing * Update retrieval of package version. (#2161) Co-authored-by: Gavin Evans <[email protected]> * MOBT-290 Environment upgrade: modal categories (#2157) * Accommodate type overflow in modal aggregator whilst resetting the data type at the end of the process. * Style fix * Move dtype setting out of numpy.clip call as this causes a casting error. (#2158) * Enforce float32 type for accumulated precipitation in nowcast accumulate code. (#2159) * Remove timezone delocalisation from estimate-emos-from-table CLI to allow correct filtering of pyarrow parquet files. (#2156) * Update checksums for acceptance test data following all of the environment upgrade work. (#2160) * Cloud condensation level updates for the new environment (#2168) * update checksums for cloud condensation level * Limits cloud base pressure and temperature to surface values if super-saturated --------- Co-authored-by: Stephen Moseley <[email protected]> --------- Co-authored-by: gavinevans <[email protected]> Co-authored-by: mspelman07 <[email protected]> Co-authored-by: Max <[email protected]> Co-authored-by: Stephen Moseley <[email protected]> Co-authored-by: Ben Hooper <[email protected]> Co-authored-by: Rachael Esler <[email protected]> * Environment upgrade tidy-up (#2171) * Remove references to python 3.6 and 3.7. Upgrade pre-commit CI environment to use python 3.12. Update readme environment building advice, though this may be at odds with the currently available version of improver in the conda library. Update the python version badge shown in the readme. * Test removal of non-commit-contributing author. * Fix readthedocs (#2172) * Fix to the readthedocs environment. * Use pip for cf_units as no versions beyong 2.0.1 available from conda-forge. * Use latest FRT option for cube combiner (#2174) * Add use_latest_frt option to the cube combiner. * Doc-string changes requested. * Added ruff flake8-bandit to pre-commit (#2145) * Removed bespoking of the wind gust metadata, e.g. typical gusts and extreme gusts (#2176) * Removed bespoking of the metadata, e.g. typical gusts and extreme gusts * Update to checksums following changes * Remove Max's erroneously added site-init file from this PR. --------- Co-authored-by: benjamin.ayliffe <[email protected]> * rebadge_forecasts_as_latest_cycle modified for use in the mix suite (#2177) * Modify rebadge_forecasts_as_latest_cycle function to look for and update a blend_time if it is present on the cubes being rebadged. This ensures that the blend time and forecast reference time cannot get out of step. * Add raises section to doc-string. * Bump actions/checkout from 4 to 5 (#2180) Bumps [actions/checkout](https://github.com/actions/checkout) from 4 to 5. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v4...v5) --- updated-dependencies: - dependency-name: actions/checkout dependency-version: '5' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Add in ability to return midpoint of bounds (#2173) * Add in ability to return midpoint of bounds * small updates to comments * Fix empty array check (#2179) * HailFraction handles masked CTT data as "no convection" (#2181) * Extends HailFraction unit test to show that masked CTT data is not treated as "no convection" * Fixes code to explicitly handle masked or invalid convective cloud top temperature values as non-convective. * Updates unit test to show that the test succeeds if the convective cloud top data are a masked array or not. * Renames cct variable for better consistency * Mobt775: Add SAMOS training plugins and unit tests (#2126) * Create statistical.py. Add functionality to fit GAMs using pyGAM. * Create GAMPredict class. Rename FitGAM to GAMFit. Add doc-strings and type-hinting for both classes. Add unit tests for both classes. * Fix doc-string typos. Run black to check code formatting. * Move pygam imports in to Classes/tests to reduce depenendency on this package. * Rename ensemble_calibration to emos_calibration and update all references. * Create samos_calibration.py and create TrainGAMsForSAMOS class within it. * Add tests for TrainGAMsForSAMOS plugin. Modify TrainGAMsForSAMOS plugin and add calculate_cube_statistics method. * Extended calculate_cube_statistics method to handle rolling window calculation over time coordinate. Add tests for this method. * Create functions for converting between cube and dataframe representations for SAMOS. Move generic helper functions for SAMOS unit tests into their own file. Create TrainEMOSForSAMOS class. Create additional unit tests for TrainGAMsForSAMOS. Add scipy monkey patch to pygam imports to work around a known bug. * Improve samos_calibration tests helper functions. Refactor TrainEMOSForSAMOS to use 2 methods which include all functionality. Add unit tests for TrainEMOSForSAMOS. Minor update to CalculateClimateAnomalies. * Modify CalculateClimateAnomalies plugin to correctly calculate the reference_epoch coordinate bounds when the inputs have multiple time points. * Make tests for TrainGAMsForSAMOS and TrainEMOSForSAMOS simpler to understand. Improve test of TrainGAMsForSAMOS by predicting from the fitted GAM and comparing the predictions to expected output. * Improve doc-strings and type hints. Move test helper functions to relevant file where there are only used in one file. Improve argument names for some test helper functions. Run pre-commit to fix formatting errors. * Correct filepath for EMOS in documentation. * Formatting changes. * Improvements to doc-strings and other changes following first review. * Changes following review. Largest change is addition of calculate_statistic_by_rolling_window method to TrainGAMsForSAMOS class. * Changes following review. Make calculate_statistic_by_rolling_window more robust to missing time points and allow it to handle period diagnostics. Add new tests. * Start using collapse_realizations methods in improver.utilities.cube_manipulation.py instead of iris method. Remove use of cell methods in TrainGAMsForSAMOS as they are unnecessary. * Minor changes following review. * Mark tests requiring pygam as skippable if pygam is not available in the environment. * Add type hints for apply_aggregator method of TrainGAMsForSAMOS class. * Fix gh action Sphinx-Pytest-Coverage: updated intersphinx inventory URLs (#2186) * new url * Update conf.py * Update doc/source/conf.py Co-authored-by: bayliffe <[email protected]> * Update conf.py --------- Co-authored-by: bayliffe <[email protected]> * Bump actions/stale from 9 to 10 (#2184) Bumps [actions/stale](https://github.com/actions/stale) from 9 to 10. - [Release notes](https://github.com/actions/stale/releases) - [Changelog](https://github.com/actions/stale/blob/main/CHANGELOG.md) - [Commits](actions/stale@v9...v10) --- updated-dependencies: - dependency-name: actions/stale dependency-version: '10' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump actions/setup-python from 5 to 6 (#2185) Bumps [actions/setup-python](https://github.com/actions/setup-python) from 5 to 6. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](actions/setup-python@v5...v6) --- updated-dependencies: - dependency-name: actions/setup-python dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * EPPT-2305 contrails engine mixing ratio (#2125) * Adding initial class setup for CondensationTrailFormation, with mixing ratio calculation method. * Update to CondensationTrailFormation and adding tests * Changes to my temporary return to keep tests passing * Adding missing test check for mixing ratio values, and adding actual engine factors * Allowing passing of bespoke engine_contrail_factors * Docstring change to stop Sphinx CI error * Documentation changes from review * Removing height cube requirement and adjusting test cube creation to fully utilise set_up_variable_cube * Changes from second review * EPPT 2534 Refactors CondensationTrailFormation so that the same resul… (#2130) * EPPT 2534 Refactors CondensationTrailFormation so that the same results can be achieved with Numpy arrays and Iris Cubes. * Ruff * Removes reference to Cubes in plugin doc-string * Renames new method at reviewer suggestion * Adding first derivative of saturation vapour pressure table (#2132) * First working version of the new SVP derivative code * Removing print statements and final changes following testing * Adding unit tests for the new SaturatedVapourPressureTableDerivative class * Adding extended documentation * Correcting path to the extended documentation * Adding Derivative to the call so the correct function is used * removing blank line in the main doc string * Refactors the two svp generator plugins to reduce duplication * Updating cube name * Adding three more unit tests to check that an error message is returned if the input temperatures to the saturated vapour pressure calculation are out of bounds. * Ruff changes * Changes following Marcus's code review * Changing the svp to svp_derivative to make it more meaningful. Also, correcting the values in one of the unit tests now that derivative values above the triple point have changes very slightly following the removal of a pair of brackets in the n2 calculation. * forcing a pre-commit hook * Pre-commit changes --------- Co-authored-by: Stephen Moseley <[email protected]> * EPPT-2389: Calculation of localised svp (#2164) * Adding almost working method with confusing difference between test calculated versions * Forcing return shape of local vapour pressure * Fixing weird precision difference due to np.float32 and np.array(,dtype=np.float32) * Fixing half of problem with pressure levels test * Fixing axis problem for pressure levels by reshaping array when calculating local vapour pressure * Generalising pressure levels shape in local_vapour_pressure calculation to allow for any shape of temperature etc data to be parsed * Revert "EPPT-2389: Calculation of localised svp (#2164)" (#2169) This reverts commit 873c009. * EPPT-2510: Adding calculation of saturation vapour pressure derivative in air (#2165) * Code changes to add the SVP derivative corrected for in air * Adding a missing temperature term to the correction calculation * Updating expected values in unit tests and changing one of the imports so it correctly finds the function * Updating one of the ecpected values within the unit tests * EPPT-2389: Calculation of localised SVP (version 2) (#2170) * Adding almost working method with confusing difference between test calculated versions * Forcing return shape of local vapour pressure * Fixing weird precision difference due to np.float32 and np.array(,dtype=np.float32) * Fixing half of problem with pressure levels test * Fixing axis problem for pressure levels by reshaping array when calculating local vapour pressure * Generalising pressure levels shape in local_vapour_pressure calculation to allow for any shape of temperature etc data to be parsed * Fixing test that is broken by environment update --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: mspelman07 <[email protected]> Co-authored-by: Stephen Moseley <[email protected]> Co-authored-by: Belinda Trotta <[email protected]> Co-authored-by: Belinda Trotta <[email protected]> Co-authored-by: bayliffe <[email protected]> Co-authored-by: gavinevans <[email protected]> Co-authored-by: Max <[email protected]> Co-authored-by: Ben Hooper <[email protected]> Co-authored-by: Rachael Esler <[email protected]> Co-authored-by: Carwyn Pelley <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: James Canvin <[email protected]> Co-authored-by: Robert Neal <[email protected]>
mo-philrelton
added a commit
that referenced
this pull request
Sep 8, 2025
* EPPT-2305 contrails engine mixing ratio (#2125) * Adding initial class setup for CondensationTrailFormation, with mixing ratio calculation method. * Update to CondensationTrailFormation and adding tests * Changes to my temporary return to keep tests passing * Adding missing test check for mixing ratio values, and adding actual engine factors * Allowing passing of bespoke engine_contrail_factors * Docstring change to stop Sphinx CI error * Documentation changes from review * Removing height cube requirement and adjusting test cube creation to fully utilise set_up_variable_cube * Changes from second review * EPPT 2534 Refactors CondensationTrailFormation so that the same resul… (#2130) * EPPT 2534 Refactors CondensationTrailFormation so that the same results can be achieved with Numpy arrays and Iris Cubes. * Ruff * Removes reference to Cubes in plugin doc-string * Renames new method at reviewer suggestion * Adding first derivative of saturation vapour pressure table (#2132) * First working version of the new SVP derivative code * Removing print statements and final changes following testing * Adding unit tests for the new SaturatedVapourPressureTableDerivative class * Adding extended documentation * Correcting path to the extended documentation * Adding Derivative to the call so the correct function is used * removing blank line in the main doc string * Refactors the two svp generator plugins to reduce duplication * Updating cube name * Adding three more unit tests to check that an error message is returned if the input temperatures to the saturated vapour pressure calculation are out of bounds. * Ruff changes * Changes following Marcus's code review * Changing the svp to svp_derivative to make it more meaningful. Also, correcting the values in one of the unit tests now that derivative values above the triple point have changes very slightly following the removal of a pair of brackets in the n2 calculation. * forcing a pre-commit hook * Pre-commit changes --------- Co-authored-by: Stephen Moseley <[email protected]> * EPPT-2389: Calculation of localised svp (#2164) * Adding almost working method with confusing difference between test calculated versions * Forcing return shape of local vapour pressure * Fixing weird precision difference due to np.float32 and np.array(,dtype=np.float32) * Fixing half of problem with pressure levels test * Fixing axis problem for pressure levels by reshaping array when calculating local vapour pressure * Generalising pressure levels shape in local_vapour_pressure calculation to allow for any shape of temperature etc data to be parsed * Revert "EPPT-2389: Calculation of localised svp (#2164)" (#2169) This reverts commit 873c009. * EPPT-2510: Adding calculation of saturation vapour pressure derivative in air (#2165) * Code changes to add the SVP derivative corrected for in air * Adding a missing temperature term to the correction calculation * Updating expected values in unit tests and changing one of the imports so it correctly finds the function * Updating one of the ecpected values within the unit tests * EPPT-2389: Calculation of localised SVP (version 2) (#2170) * Adding almost working method with confusing difference between test calculated versions * Forcing return shape of local vapour pressure * Fixing weird precision difference due to np.float32 and np.array(,dtype=np.float32) * Fixing half of problem with pressure levels test * Fixing axis problem for pressure levels by reshaping array when calculating local vapour pressure * Generalising pressure levels shape in local_vapour_pressure calculation to allow for any shape of temperature etc data to be parsed * Fixing test that is broken by environment update --------- Co-authored-by: Stephen Moseley <[email protected]> Co-authored-by: Robert Neal <[email protected]>
ryan-cocking-mo
pushed a commit
that referenced
this pull request
Nov 13, 2025
* EPPT-2305 contrails engine mixing ratio (#2125) * Adding initial class setup for CondensationTrailFormation, with mixing ratio calculation method. * Update to CondensationTrailFormation and adding tests * Changes to my temporary return to keep tests passing * Adding missing test check for mixing ratio values, and adding actual engine factors * Allowing passing of bespoke engine_contrail_factors * Docstring change to stop Sphinx CI error * Documentation changes from review * Removing height cube requirement and adjusting test cube creation to fully utilise set_up_variable_cube * Changes from second review * EPPT 2534 Refactors CondensationTrailFormation so that the same resul… (#2130) * EPPT 2534 Refactors CondensationTrailFormation so that the same results can be achieved with Numpy arrays and Iris Cubes. * Ruff * Removes reference to Cubes in plugin doc-string * Renames new method at reviewer suggestion * Adding first derivative of saturation vapour pressure table (#2132) * First working version of the new SVP derivative code * Removing print statements and final changes following testing * Adding unit tests for the new SaturatedVapourPressureTableDerivative class * Adding extended documentation * Correcting path to the extended documentation * Adding Derivative to the call so the correct function is used * removing blank line in the main doc string * Refactors the two svp generator plugins to reduce duplication * Updating cube name * Adding three more unit tests to check that an error message is returned if the input temperatures to the saturated vapour pressure calculation are out of bounds. * Ruff changes * Changes following Marcus's code review * Changing the svp to svp_derivative to make it more meaningful. Also, correcting the values in one of the unit tests now that derivative values above the triple point have changes very slightly following the removal of a pair of brackets in the n2 calculation. * forcing a pre-commit hook * Pre-commit changes --------- Co-authored-by: Stephen Moseley <[email protected]> * EPPT-2389: Calculation of localised svp (#2164) * Adding almost working method with confusing difference between test calculated versions * Forcing return shape of local vapour pressure * Fixing weird precision difference due to np.float32 and np.array(,dtype=np.float32) * Fixing half of problem with pressure levels test * Fixing axis problem for pressure levels by reshaping array when calculating local vapour pressure * Generalising pressure levels shape in local_vapour_pressure calculation to allow for any shape of temperature etc data to be parsed * Revert "EPPT-2389: Calculation of localised svp (#2164)" (#2169) This reverts commit 873c009. * EPPT-2510: Adding calculation of saturation vapour pressure derivative in air (#2165) * Code changes to add the SVP derivative corrected for in air * Adding a missing temperature term to the correction calculation * Updating expected values in unit tests and changing one of the imports so it correctly finds the function * Updating one of the ecpected values within the unit tests * EPPT-2389: Calculation of localised SVP (version 2) (#2170) * Adding almost working method with confusing difference between test calculated versions * Forcing return shape of local vapour pressure * Fixing weird precision difference due to np.float32 and np.array(,dtype=np.float32) * Fixing half of problem with pressure levels test * Fixing axis problem for pressure levels by reshaping array when calculating local vapour pressure * Generalising pressure levels shape in local_vapour_pressure calculation to allow for any shape of temperature etc data to be parsed * Fixing test that is broken by environment update --------- Co-authored-by: Stephen Moseley <[email protected]> Co-authored-by: Robert Neal <[email protected]>
ryan-cocking-mo
added a commit
that referenced
this pull request
Nov 14, 2025
* EPPT-2657: Rebasing the condensation trails development branch (#2189) * EPPT-2305 contrails engine mixing ratio (#2125) * Adding initial class setup for CondensationTrailFormation, with mixing ratio calculation method. * Update to CondensationTrailFormation and adding tests * Changes to my temporary return to keep tests passing * Adding missing test check for mixing ratio values, and adding actual engine factors * Allowing passing of bespoke engine_contrail_factors * Docstring change to stop Sphinx CI error * Documentation changes from review * Removing height cube requirement and adjusting test cube creation to fully utilise set_up_variable_cube * Changes from second review * EPPT 2534 Refactors CondensationTrailFormation so that the same resul… (#2130) * EPPT 2534 Refactors CondensationTrailFormation so that the same results can be achieved with Numpy arrays and Iris Cubes. * Ruff * Removes reference to Cubes in plugin doc-string * Renames new method at reviewer suggestion * Adding first derivative of saturation vapour pressure table (#2132) * First working version of the new SVP derivative code * Removing print statements and final changes following testing * Adding unit tests for the new SaturatedVapourPressureTableDerivative class * Adding extended documentation * Correcting path to the extended documentation * Adding Derivative to the call so the correct function is used * removing blank line in the main doc string * Refactors the two svp generator plugins to reduce duplication * Updating cube name * Adding three more unit tests to check that an error message is returned if the input temperatures to the saturated vapour pressure calculation are out of bounds. * Ruff changes * Changes following Marcus's code review * Changing the svp to svp_derivative to make it more meaningful. Also, correcting the values in one of the unit tests now that derivative values above the triple point have changes very slightly following the removal of a pair of brackets in the n2 calculation. * forcing a pre-commit hook * Pre-commit changes --------- Co-authored-by: Stephen Moseley <[email protected]> * EPPT-2389: Calculation of localised svp (#2164) * Adding almost working method with confusing difference between test calculated versions * Forcing return shape of local vapour pressure * Fixing weird precision difference due to np.float32 and np.array(,dtype=np.float32) * Fixing half of problem with pressure levels test * Fixing axis problem for pressure levels by reshaping array when calculating local vapour pressure * Generalising pressure levels shape in local_vapour_pressure calculation to allow for any shape of temperature etc data to be parsed * Revert "EPPT-2389: Calculation of localised svp (#2164)" (#2169) This reverts commit 873c009. * EPPT-2510: Adding calculation of saturation vapour pressure derivative in air (#2165) * Code changes to add the SVP derivative corrected for in air * Adding a missing temperature term to the correction calculation * Updating expected values in unit tests and changing one of the imports so it correctly finds the function * Updating one of the ecpected values within the unit tests * EPPT-2389: Calculation of localised SVP (version 2) (#2170) * Adding almost working method with confusing difference between test calculated versions * Forcing return shape of local vapour pressure * Fixing weird precision difference due to np.float32 and np.array(,dtype=np.float32) * Fixing half of problem with pressure levels test * Fixing axis problem for pressure levels by reshaping array when calculating local vapour pressure * Generalising pressure levels shape in local_vapour_pressure calculation to allow for any shape of temperature etc data to be parsed * Fixing test that is broken by environment update --------- Co-authored-by: Stephen Moseley <[email protected]> Co-authored-by: Robert Neal <[email protected]> * EPPT-2642 contrails force svp table to return water only (#2191) * Adding first derivative of saturation vapour pressure table (#2132) * First working version of the new SVP derivative code * Removing print statements and final changes following testing * Adding unit tests for the new SaturatedVapourPressureTableDerivative class * Adding extended documentation * Correcting path to the extended documentation * Adding Derivative to the call so the correct function is used * removing blank line in the main doc string * Refactors the two svp generator plugins to reduce duplication * Updating cube name * Adding three more unit tests to check that an error message is returned if the input temperatures to the saturated vapour pressure calculation are out of bounds. * Ruff changes * Changes following Marcus's code review * Changing the svp to svp_derivative to make it more meaningful. Also, correcting the values in one of the unit tests now that derivative values above the triple point have changes very slightly following the removal of a pair of brackets in the n2 calculation. * forcing a pre-commit hook * Pre-commit changes --------- Co-authored-by: Stephen Moseley <[email protected]> * Revert "EPPT-2389: Calculation of localised svp (#2164)" (#2169) This reverts commit 873c009. * EPPT-2389: Calculation of localised SVP (version 2) (#2170) * Adding almost working method with confusing difference between test calculated versions * Forcing return shape of local vapour pressure * Fixing weird precision difference due to np.float32 and np.array(,dtype=np.float32) * Fixing half of problem with pressure levels test * Fixing axis problem for pressure levels by reshaping array when calculating local vapour pressure * Generalising pressure levels shape in local_vapour_pressure calculation to allow for any shape of temperature etc data to be parsed * Added name to contributors and mailmap * Added optional flags to SVP and SVP derivative lookup tables, to construct tables with respect to water or ice only. * Additional unit tests for SVP and SVP derivative lookup tables * Pythonic bool check and pre-commit * Added extra warnings if constructing SVP tables for water or ice outside valid temperatures * Refactored unit tests to account for additional warnings * Update docstrings with temperature ranges for water and ice * Pre-commit changes * Minor additions to docstring and warnings about validity of Goff-Gratch in water and ice * Skip offending test in SVP table, to be re-enabled once latest IMPROVER environment is available * test_basic for SVP table now runs * Changed name of SVP derivative table class --------- Co-authored-by: Robert Neal <[email protected]> Co-authored-by: Stephen Moseley <[email protected]> Co-authored-by: Phil <[email protected]> * EPPT-2395 contrails critical temperatures and intercepts (#2192) * Adding first derivative of saturation vapour pressure table (#2132) * First working version of the new SVP derivative code * Removing print statements and final changes following testing * Adding unit tests for the new SaturatedVapourPressureTableDerivative class * Adding extended documentation * Correcting path to the extended documentation * Adding Derivative to the call so the correct function is used * removing blank line in the main doc string * Refactors the two svp generator plugins to reduce duplication * Updating cube name * Adding three more unit tests to check that an error message is returned if the input temperatures to the saturated vapour pressure calculation are out of bounds. * Ruff changes * Changes following Marcus's code review * Changing the svp to svp_derivative to make it more meaningful. Also, correcting the values in one of the unit tests now that derivative values above the triple point have changes very slightly following the removal of a pair of brackets in the n2 calculation. * forcing a pre-commit hook * Pre-commit changes --------- Co-authored-by: Stephen Moseley <[email protected]> * Revert "EPPT-2389: Calculation of localised svp (#2164)" (#2169) This reverts commit 873c009. * EPPT-2389: Calculation of localised SVP (version 2) (#2170) * Adding almost working method with confusing difference between test calculated versions * Forcing return shape of local vapour pressure * Fixing weird precision difference due to np.float32 and np.array(,dtype=np.float32) * Fixing half of problem with pressure levels test * Fixing axis problem for pressure levels by reshaping array when calculating local vapour pressure * Generalising pressure levels shape in local_vapour_pressure calculation to allow for any shape of temperature etc data to be parsed * Added initial critical temperature calculation from notebook * Vectorised critical temperature calculation * Refactoring and renaming * Doc edits and minor reformatting * Minor docstring changes * Enable SVP tables with respect to water * Fixed name of SVP table * Added type checking to critical temperatures, and reduced explicit slicing * First test for critical temperatures. Currently checks just for array shapes. * Made name of calculation more descriptive. Minor docstring edits. Refactored type checks. * Line break on error messages * Refactor test to outer loop of critical temperature calculation * Added correct docstring to critical temperature values test. Added another test that checks the exceptions that get raised by the critical temperature calculation. * Two diagrams added to contrails docs (not yet commented on) * Slightly improved return formatting in docs * Figure update * Doc update * Duplicate SVP table test removed * Removed explicit reference to num_pressure_levels in critical temperature function. Improved use of argmin. Removed an interpolation step. * Simplified critical temperature function signatures by using self, and passing in the SVP table objects themselves, rather than just the data. * Removed type checking (will be implemented in later ticket) and moved assignment to temperatures and intercepts inside the function, rather than returning. * Corrected x axis label of appleman diagram * Increase precision of rtol in unit test Co-authored-by: Stephen Moseley <[email protected]> * Remove return type from function Co-authored-by: Stephen Moseley <[email protected]> * Docstring and variable name changes from review * Changed another rtol to 1e-6 * Renamed unit test * Doc update Co-authored-by: Stephen Moseley <[email protected]> * Doc changes * Reduced precision of test inputs to match rtol (7 significant figures) --------- Co-authored-by: Robert Neal <[email protected]> Co-authored-by: Stephen Moseley <[email protected]> Co-authored-by: Phil <[email protected]> * EPPT-2390 contrail persistency logic (#2194) * Initial commit, with descriptive conditions and an extensive arg list * Conditions method finished. Added extra docs via RST file. * Basic test for contrails persistency * Minor doc edit * Finished value test for contrails persistency. Takes in 16 input values, to cover the 16 unique combinations of conditions in the contrail persistency function. * Contrails tests now has a shape for the expected output array in which each dimension has a different size. Test docstring updated. * Typo * Introduced small function within conditional logic that broadcasts arrays to larger shapes, allowing for more flexible logical comparisons, i.e. critical temperatures may only have three dimensions instead of four * Minor refactor * Docstring update * Docstring update * Renamed first persistency test to emphasise that it checks combinations of formation conditions. Added second test that checks the shapes of input and output arrays. * Minor docstring update * Removed explicit mention of type in docstring. Not only does the sphinx documentation generated this automatically anyway (making the explicit mention redundant), but by omitting e.g. '(np.ndarray)' in the docstring, sphinx is able to look at the type hint in the function signature and create a hyperlink to the numpy documentation. * Add more physical test inputs * adjust local vp in test * conditions numbers in code * typo * Swapped outputs of persistency function. Replaced unphysical values in shapes test with realistic ones. * pre-commit * EPPT-2784 contrails local vapour pressure water only (#2216) * Added a flag to pass through 'water_only' or 'ice_only' flags to the construction of the SVP table. * Updated local VP calculation and test to use values w.r.t. water * Added two unit tests for calculating svp in air with respect to water or ice only * EPP 2443: Adding a new ContrailsHeightExtractor class and associated unit tests (#2206) * Adding a new ContrailsHeightExtractor class and associated unit tests. * Changes following a pre-commit hook * Removing excessive code comments * More pre-commit hook changes * Changes to the docstrings and remove excessive comments. * Removing information from the Class docstring that is duplicated in the main process function. * Updating the main class and unit tests to use the attributes within the formation_cube * Removing certain attributes from the template cube. * Update improver/psychrometric_calculations/condensation_trails.py Co-authored-by: ryan-cocking-mo <[email protected]> * Update improver/psychrometric_calculations/condensation_trails.py Co-authored-by: ryan-cocking-mo <[email protected]> * Update improver/psychrometric_calculations/condensation_trails.py Co-authored-by: ryan-cocking-mo <[email protected]> * Update improver/psychrometric_calculations/condensation_trails.py Co-authored-by: ryan-cocking-mo <[email protected]> * Update improver/psychrometric_calculations/condensation_trails.py Co-authored-by: ryan-cocking-mo <[email protected]> * Update improver/psychrometric_calculations/condensation_trails.py Co-authored-by: ryan-cocking-mo <[email protected]> * Update improver/psychrometric_calculations/condensation_trails.py Co-authored-by: ryan-cocking-mo <[email protected]> * Update improver_tests/psychrometric_calculations/condensation_trails/test_ContrailHeightExtractor.py Co-authored-by: ryan-cocking-mo <[email protected]> * Update improver_tests/psychrometric_calculations/condensation_trails/test_ContrailHeightExtractor.py Co-authored-by: ryan-cocking-mo <[email protected]> * Update improver_tests/psychrometric_calculations/condensation_trails/test_ContrailHeightExtractor.py Co-authored-by: ryan-cocking-mo <[email protected]> * Changes following Ryan's code review * Fixing the running of the unit tests * Changes to one of the tests so parametrize is used more efficiently * Added height extractor class to API --------- Co-authored-by: ryan-cocking-mo <[email protected]> * EPPT-2721 contrails output categorical data (#2209) * Function to output categorical (integer) array from input boolean arrays * Unit test * Initial function to generate categorical cube * Initial test for categorical cube output * Categorical cube test returns expected shape and data * Check cube contents and metadata in test * todo * Refactoring of class to reduce argument lists, increase use of self, and tidy up docstrings * Changes to unit tests to account for function signature changes. Because SVP ice table is now used within the persistency function, this required changing the parametrisation of the combination test. Every unique combination (16 total) now cannot be tested easily, so we have settled for most of the combinations. * Array shape checking in process_from_arrays * Cube input error checks. Minor refactoring of input array checks. * minor edit * Update categorical metadata * Category check added to cube output unit test * remove print * Cube input check test * Initial version of test to check that pressure, temperature and relative humidity inputs can produce the expected result from an Appleman diagram. * Modify test to show in-progress. To be completed once work from contrail persistency fix has been merged. * Added contrail class to API * Changed categorical output dtype to int32 * End-to-end test that checks values of output categorical cube, from input temperature and humidity cubes * Changed hard-coded freezing temperature to use absolute zero constant * EPPT-2736 contrail height extractor entry point (#2226) * Skeleton of process_from_arrays method with docstring * Refactored process to move calculations into process_from_arrays. Added array checks. Renamed cube creation method. * Line breaks * typehint * minor doc format * added process_from_arrays call to minmax values test * minor format of error checking * process_from_arrays error check test --------- Co-authored-by: Phil <[email protected]> Co-authored-by: Stephen Moseley <[email protected]> Co-authored-by: Robert Neal <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
EPPT-2510
Description
This change is adding the calculation of saturation vapour pressure derivative in air.
The saturation vapour pressure (SVP), SVP corrected for air, and SVP derivative are already calculated by existing functions, so this change is focused on correcting the SVP derivative so it is valid in air. This involves creating the following new functions within
psychrometric_calculations.py:_svp_derivative_table- to run the extistingSaturatedVapourPressureTableDerivativefunction and generate a table of SVP derivative values._svp_derivative_from_lookup- To look up the SVP derivative values from the table of SVP derivative values, based in input temperatures.calculate_svp_derivative_in_air- This is modifying the original SVP derivative values by correcting them so they are valid in air.This change has also added a new unit test script called
test_calculate_svp_derivative_in_air.pyto test the new functions listed in Points 2 and 3 above. A unit test has not been added for_svp_derivative_tableas this just callsSaturatedVapourPressureTableDerivativewhich is already covered by unit tests in theimprover_tests/generate_ancillariesdirectory.Testing: