EPPT-2390 contrail persistency logic #2194
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…es, to cover the 16 unique combinations of conditions in the contrail persistency function.
…h each dimension has a different size. Test docstring updated.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## EPPT_2072_contrails_design #2194 +/- ##
=============================================================
Coverage ? 98.46%
=============================================================
Files ? 142
Lines ? 13982
Branches ? 0
=============================================================
Hits ? 13767
Misses ? 215
Partials ? 0 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
…rays to larger shapes, allowing for more flexible logical comparisons, i.e. critical temperatures may only have three dimensions instead of four
…ns of formation conditions. Added second test that checks the shapes of input and output arrays.
…nx documentation generated this automatically anyway (making the explicit mention redundant), but by omitting e.g. '(np.ndarray)' in the docstring, sphinx is able to look at the type hint in the function signature and create a hyperlink to the numpy documentation.
mo-robert-purvis
approved these changes
Oct 2, 2025
MoseleyS
requested changes
Oct 3, 2025
Member
MoseleyS
left a comment
MoseleyS
left a comment
There was a problem hiding this comment.
I'm struggling to decide whether the maths is correct, although I haven't seen anything to say it isn't.
...tended_documentation/psychrometric_calculations/condensation_trails/formation_conditions.rst
Outdated
Show resolved
Hide resolved
...over_tests/psychrometric_calculations/condensation_trails/test_CondensationTrailFormation.py
Outdated
Show resolved
Hide resolved
MoseleyS
approved these changes
Oct 7, 2025
Member
MoseleyS
left a comment
MoseleyS
left a comment
There was a problem hiding this comment.
I approve of the changes, but conflicts with the base branch need resolving before this can be merged.
…n shapes test with realistic ones.
ryan-cocking-mo
merged commit b8550fb
into
metoppv:EPPT_2072_contrails_design
7 checks passed
ryan-cocking-mo
added a commit
that referenced
this pull request
Nov 13, 2025
* Initial commit, with descriptive conditions and an extensive arg list * Conditions method finished. Added extra docs via RST file. * Basic test for contrails persistency * Minor doc edit * Finished value test for contrails persistency. Takes in 16 input values, to cover the 16 unique combinations of conditions in the contrail persistency function. * Contrails tests now has a shape for the expected output array in which each dimension has a different size. Test docstring updated. * Typo * Introduced small function within conditional logic that broadcasts arrays to larger shapes, allowing for more flexible logical comparisons, i.e. critical temperatures may only have three dimensions instead of four * Minor refactor * Docstring update * Docstring update * Renamed first persistency test to emphasise that it checks combinations of formation conditions. Added second test that checks the shapes of input and output arrays. * Minor docstring update * Removed explicit mention of type in docstring. Not only does the sphinx documentation generated this automatically anyway (making the explicit mention redundant), but by omitting e.g. '(np.ndarray)' in the docstring, sphinx is able to look at the type hint in the function signature and create a hyperlink to the numpy documentation. * Add more physical test inputs * adjust local vp in test * conditions numbers in code * typo * Swapped outputs of persistency function. Replaced unphysical values in shapes test with realistic ones. * pre-commit
ryan-cocking-mo
added a commit
that referenced
this pull request
Nov 14, 2025
* EPPT-2657: Rebasing the condensation trails development branch (#2189) * EPPT-2305 contrails engine mixing ratio (#2125) * Adding initial class setup for CondensationTrailFormation, with mixing ratio calculation method. * Update to CondensationTrailFormation and adding tests * Changes to my temporary return to keep tests passing * Adding missing test check for mixing ratio values, and adding actual engine factors * Allowing passing of bespoke engine_contrail_factors * Docstring change to stop Sphinx CI error * Documentation changes from review * Removing height cube requirement and adjusting test cube creation to fully utilise set_up_variable_cube * Changes from second review * EPPT 2534 Refactors CondensationTrailFormation so that the same resul… (#2130) * EPPT 2534 Refactors CondensationTrailFormation so that the same results can be achieved with Numpy arrays and Iris Cubes. * Ruff * Removes reference to Cubes in plugin doc-string * Renames new method at reviewer suggestion * Adding first derivative of saturation vapour pressure table (#2132) * First working version of the new SVP derivative code * Removing print statements and final changes following testing * Adding unit tests for the new SaturatedVapourPressureTableDerivative class * Adding extended documentation * Correcting path to the extended documentation * Adding Derivative to the call so the correct function is used * removing blank line in the main doc string * Refactors the two svp generator plugins to reduce duplication * Updating cube name * Adding three more unit tests to check that an error message is returned if the input temperatures to the saturated vapour pressure calculation are out of bounds. * Ruff changes * Changes following Marcus's code review * Changing the svp to svp_derivative to make it more meaningful. Also, correcting the values in one of the unit tests now that derivative values above the triple point have changes very slightly following the removal of a pair of brackets in the n2 calculation. * forcing a pre-commit hook * Pre-commit changes --------- Co-authored-by: Stephen Moseley <stephen.moseley@metoffice.gov.uk> * EPPT-2389: Calculation of localised svp (#2164) * Adding almost working method with confusing difference between test calculated versions * Forcing return shape of local vapour pressure * Fixing weird precision difference due to np.float32 and np.array(,dtype=np.float32) * Fixing half of problem with pressure levels test * Fixing axis problem for pressure levels by reshaping array when calculating local vapour pressure * Generalising pressure levels shape in local_vapour_pressure calculation to allow for any shape of temperature etc data to be parsed * Revert "EPPT-2389: Calculation of localised svp (#2164)" (#2169) This reverts commit 873c009. * EPPT-2510: Adding calculation of saturation vapour pressure derivative in air (#2165) * Code changes to add the SVP derivative corrected for in air * Adding a missing temperature term to the correction calculation * Updating expected values in unit tests and changing one of the imports so it correctly finds the function * Updating one of the ecpected values within the unit tests * EPPT-2389: Calculation of localised SVP (version 2) (#2170) * Adding almost working method with confusing difference between test calculated versions * Forcing return shape of local vapour pressure * Fixing weird precision difference due to np.float32 and np.array(,dtype=np.float32) * Fixing half of problem with pressure levels test * Fixing axis problem for pressure levels by reshaping array when calculating local vapour pressure * Generalising pressure levels shape in local_vapour_pressure calculation to allow for any shape of temperature etc data to be parsed * Fixing test that is broken by environment update --------- Co-authored-by: Stephen Moseley <stephen.moseley@metoffice.gov.uk> Co-authored-by: Robert Neal <robert.neal@metoffice.gov.uk> * EPPT-2642 contrails force svp table to return water only (#2191) * Adding first derivative of saturation vapour pressure table (#2132) * First working version of the new SVP derivative code * Removing print statements and final changes following testing * Adding unit tests for the new SaturatedVapourPressureTableDerivative class * Adding extended documentation * Correcting path to the extended documentation * Adding Derivative to the call so the correct function is used * removing blank line in the main doc string * Refactors the two svp generator plugins to reduce duplication * Updating cube name * Adding three more unit tests to check that an error message is returned if the input temperatures to the saturated vapour pressure calculation are out of bounds. * Ruff changes * Changes following Marcus's code review * Changing the svp to svp_derivative to make it more meaningful. Also, correcting the values in one of the unit tests now that derivative values above the triple point have changes very slightly following the removal of a pair of brackets in the n2 calculation. * forcing a pre-commit hook * Pre-commit changes --------- Co-authored-by: Stephen Moseley <stephen.moseley@metoffice.gov.uk> * Revert "EPPT-2389: Calculation of localised svp (#2164)" (#2169) This reverts commit 873c009. * EPPT-2389: Calculation of localised SVP (version 2) (#2170) * Adding almost working method with confusing difference between test calculated versions * Forcing return shape of local vapour pressure * Fixing weird precision difference due to np.float32 and np.array(,dtype=np.float32) * Fixing half of problem with pressure levels test * Fixing axis problem for pressure levels by reshaping array when calculating local vapour pressure * Generalising pressure levels shape in local_vapour_pressure calculation to allow for any shape of temperature etc data to be parsed * Added name to contributors and mailmap * Added optional flags to SVP and SVP derivative lookup tables, to construct tables with respect to water or ice only. * Additional unit tests for SVP and SVP derivative lookup tables * Pythonic bool check and pre-commit * Added extra warnings if constructing SVP tables for water or ice outside valid temperatures * Refactored unit tests to account for additional warnings * Update docstrings with temperature ranges for water and ice * Pre-commit changes * Minor additions to docstring and warnings about validity of Goff-Gratch in water and ice * Skip offending test in SVP table, to be re-enabled once latest IMPROVER environment is available * test_basic for SVP table now runs * Changed name of SVP derivative table class --------- Co-authored-by: Robert Neal <robert.neal@metoffice.gov.uk> Co-authored-by: Stephen Moseley <stephen.moseley@metoffice.gov.uk> Co-authored-by: Phil <phil.relton@metoffice.gov.uk> * EPPT-2395 contrails critical temperatures and intercepts (#2192) * Adding first derivative of saturation vapour pressure table (#2132) * First working version of the new SVP derivative code * Removing print statements and final changes following testing * Adding unit tests for the new SaturatedVapourPressureTableDerivative class * Adding extended documentation * Correcting path to the extended documentation * Adding Derivative to the call so the correct function is used * removing blank line in the main doc string * Refactors the two svp generator plugins to reduce duplication * Updating cube name * Adding three more unit tests to check that an error message is returned if the input temperatures to the saturated vapour pressure calculation are out of bounds. * Ruff changes * Changes following Marcus's code review * Changing the svp to svp_derivative to make it more meaningful. Also, correcting the values in one of the unit tests now that derivative values above the triple point have changes very slightly following the removal of a pair of brackets in the n2 calculation. * forcing a pre-commit hook * Pre-commit changes --------- Co-authored-by: Stephen Moseley <stephen.moseley@metoffice.gov.uk> * Revert "EPPT-2389: Calculation of localised svp (#2164)" (#2169) This reverts commit 873c009. * EPPT-2389: Calculation of localised SVP (version 2) (#2170) * Adding almost working method with confusing difference between test calculated versions * Forcing return shape of local vapour pressure * Fixing weird precision difference due to np.float32 and np.array(,dtype=np.float32) * Fixing half of problem with pressure levels test * Fixing axis problem for pressure levels by reshaping array when calculating local vapour pressure * Generalising pressure levels shape in local_vapour_pressure calculation to allow for any shape of temperature etc data to be parsed * Added initial critical temperature calculation from notebook * Vectorised critical temperature calculation * Refactoring and renaming * Doc edits and minor reformatting * Minor docstring changes * Enable SVP tables with respect to water * Fixed name of SVP table * Added type checking to critical temperatures, and reduced explicit slicing * First test for critical temperatures. Currently checks just for array shapes. * Made name of calculation more descriptive. Minor docstring edits. Refactored type checks. * Line break on error messages * Refactor test to outer loop of critical temperature calculation * Added correct docstring to critical temperature values test. Added another test that checks the exceptions that get raised by the critical temperature calculation. * Two diagrams added to contrails docs (not yet commented on) * Slightly improved return formatting in docs * Figure update * Doc update * Duplicate SVP table test removed * Removed explicit reference to num_pressure_levels in critical temperature function. Improved use of argmin. Removed an interpolation step. * Simplified critical temperature function signatures by using self, and passing in the SVP table objects themselves, rather than just the data. * Removed type checking (will be implemented in later ticket) and moved assignment to temperatures and intercepts inside the function, rather than returning. * Corrected x axis label of appleman diagram * Increase precision of rtol in unit test Co-authored-by: Stephen Moseley <stephen.moseley@metoffice.gov.uk> * Remove return type from function Co-authored-by: Stephen Moseley <stephen.moseley@metoffice.gov.uk> * Docstring and variable name changes from review * Changed another rtol to 1e-6 * Renamed unit test * Doc update Co-authored-by: Stephen Moseley <stephen.moseley@metoffice.gov.uk> * Doc changes * Reduced precision of test inputs to match rtol (7 significant figures) --------- Co-authored-by: Robert Neal <robert.neal@metoffice.gov.uk> Co-authored-by: Stephen Moseley <stephen.moseley@metoffice.gov.uk> Co-authored-by: Phil <phil.relton@metoffice.gov.uk> * EPPT-2390 contrail persistency logic (#2194) * Initial commit, with descriptive conditions and an extensive arg list * Conditions method finished. Added extra docs via RST file. * Basic test for contrails persistency * Minor doc edit * Finished value test for contrails persistency. Takes in 16 input values, to cover the 16 unique combinations of conditions in the contrail persistency function. * Contrails tests now has a shape for the expected output array in which each dimension has a different size. Test docstring updated. * Typo * Introduced small function within conditional logic that broadcasts arrays to larger shapes, allowing for more flexible logical comparisons, i.e. critical temperatures may only have three dimensions instead of four * Minor refactor * Docstring update * Docstring update * Renamed first persistency test to emphasise that it checks combinations of formation conditions. Added second test that checks the shapes of input and output arrays. * Minor docstring update * Removed explicit mention of type in docstring. Not only does the sphinx documentation generated this automatically anyway (making the explicit mention redundant), but by omitting e.g. '(np.ndarray)' in the docstring, sphinx is able to look at the type hint in the function signature and create a hyperlink to the numpy documentation. * Add more physical test inputs * adjust local vp in test * conditions numbers in code * typo * Swapped outputs of persistency function. Replaced unphysical values in shapes test with realistic ones. * pre-commit * EPPT-2784 contrails local vapour pressure water only (#2216) * Added a flag to pass through 'water_only' or 'ice_only' flags to the construction of the SVP table. * Updated local VP calculation and test to use values w.r.t. water * Added two unit tests for calculating svp in air with respect to water or ice only * EPP 2443: Adding a new ContrailsHeightExtractor class and associated unit tests (#2206) * Adding a new ContrailsHeightExtractor class and associated unit tests. * Changes following a pre-commit hook * Removing excessive code comments * More pre-commit hook changes * Changes to the docstrings and remove excessive comments. * Removing information from the Class docstring that is duplicated in the main process function. * Updating the main class and unit tests to use the attributes within the formation_cube * Removing certain attributes from the template cube. * Update improver/psychrometric_calculations/condensation_trails.py Co-authored-by: ryan-cocking-mo <ryan.cocking@metoffice.gov.uk> * Update improver/psychrometric_calculations/condensation_trails.py Co-authored-by: ryan-cocking-mo <ryan.cocking@metoffice.gov.uk> * Update improver/psychrometric_calculations/condensation_trails.py Co-authored-by: ryan-cocking-mo <ryan.cocking@metoffice.gov.uk> * Update improver/psychrometric_calculations/condensation_trails.py Co-authored-by: ryan-cocking-mo <ryan.cocking@metoffice.gov.uk> * Update improver/psychrometric_calculations/condensation_trails.py Co-authored-by: ryan-cocking-mo <ryan.cocking@metoffice.gov.uk> * Update improver/psychrometric_calculations/condensation_trails.py Co-authored-by: ryan-cocking-mo <ryan.cocking@metoffice.gov.uk> * Update improver/psychrometric_calculations/condensation_trails.py Co-authored-by: ryan-cocking-mo <ryan.cocking@metoffice.gov.uk> * Update improver_tests/psychrometric_calculations/condensation_trails/test_ContrailHeightExtractor.py Co-authored-by: ryan-cocking-mo <ryan.cocking@metoffice.gov.uk> * Update improver_tests/psychrometric_calculations/condensation_trails/test_ContrailHeightExtractor.py Co-authored-by: ryan-cocking-mo <ryan.cocking@metoffice.gov.uk> * Update improver_tests/psychrometric_calculations/condensation_trails/test_ContrailHeightExtractor.py Co-authored-by: ryan-cocking-mo <ryan.cocking@metoffice.gov.uk> * Changes following Ryan's code review * Fixing the running of the unit tests * Changes to one of the tests so parametrize is used more efficiently * Added height extractor class to API --------- Co-authored-by: ryan-cocking-mo <ryan.cocking@metoffice.gov.uk> * EPPT-2721 contrails output categorical data (#2209) * Function to output categorical (integer) array from input boolean arrays * Unit test * Initial function to generate categorical cube * Initial test for categorical cube output * Categorical cube test returns expected shape and data * Check cube contents and metadata in test * todo * Refactoring of class to reduce argument lists, increase use of self, and tidy up docstrings * Changes to unit tests to account for function signature changes. Because SVP ice table is now used within the persistency function, this required changing the parametrisation of the combination test. Every unique combination (16 total) now cannot be tested easily, so we have settled for most of the combinations. * Array shape checking in process_from_arrays * Cube input error checks. Minor refactoring of input array checks. * minor edit * Update categorical metadata * Category check added to cube output unit test * remove print * Cube input check test * Initial version of test to check that pressure, temperature and relative humidity inputs can produce the expected result from an Appleman diagram. * Modify test to show in-progress. To be completed once work from contrail persistency fix has been merged. * Added contrail class to API * Changed categorical output dtype to int32 * End-to-end test that checks values of output categorical cube, from input temperature and humidity cubes * Changed hard-coded freezing temperature to use absolute zero constant * EPPT-2736 contrail height extractor entry point (#2226) * Skeleton of process_from_arrays method with docstring * Refactored process to move calculations into process_from_arrays. Added array checks. Renamed cube creation method. * Line breaks * typehint * minor doc format * added process_from_arrays call to minmax values test * minor format of error checking * process_from_arrays error check test --------- Co-authored-by: Phil <phil.relton@metoffice.gov.uk> Co-authored-by: Stephen Moseley <stephen.moseley@metoffice.gov.uk> Co-authored-by: Robert Neal <robert.neal@metoffice.gov.uk>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Parent ticket: https://metoffice.atlassian.net/browse/EPPT-2390
Child ticket: https://metoffice.atlassian.net/browse/EPPT-2671
This adds conditional logic for whether persistent or non-persistent contrails will form, and unit tests.
The conditional logic currently outputs two arrays for nonpersistent and persistent contrails.
No new code was added to the
process()orprocess_from_arrays()methods, as certain quantities such as the critical temperature have not yet been merged with this work.