Skip to content

Conversation

@bayliffe
Copy link
Contributor

@bayliffe bayliffe commented Jul 7, 2025

Related to https://github.com/metoppv/mo-blue-team/issues/942

This PR updates the acceptance test data for batch 8. Changes are shown below.

Data comparison

data/nowcast-extrapolate/extrapolate/kgo.nc
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : forecast_reference_time : 9 <> 8 : VALUES : "gregorian" : "standard"
=============================================
data/nowcast-extrapolate/metadata/kgo_with_metadata.nc
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : forecast_reference_time : 9 <> 8 : VALUES : "gregorian" : "standard"
=============================================
data/nowcast-optical-flow-from-winds/kgo_15min.nc
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : VARIABLE : precipitation_advection_x_velocity : POSITION : [20,22] : VALUES : 12.9621 <> 12.9621
DIFFER : VARIABLE : precipitation_advection_x_velocity : POSITION : [39,14] : VALUES : 11.0426 <> 11.0426
DIFFER : VARIABLE : precipitation_advection_x_velocity : POSITION : [56,53] : VALUES : 14.5031 <> 14.5031
DIFFER : VARIABLE : precipitation_advection_x_velocity : POSITION : [57,18] : VALUES : 11.9088 <> 11.9088
DIFFER : VARIABLE : precipitation_advection_x_velocity : POSITION : [70,56] : VALUES : 13.522 <> 13.522
DIFFER : VARIABLE : precipitation_advection_y_velocity : POSITION : [67,87] : VALUES : -14.3618 <> -14.3618
DIFFER : VARIABLE : precipitation_advection_y_velocity : POSITION : [71,93] : VALUES : -15.742 <> -15.742
DIFFER : VARIABLE : precipitation_advection_y_velocity : POSITION : [72,77] : VALUES : -15.0133 <> -15.0133
DIFFER : VARIABLE : precipitation_advection_y_velocity : POSITION : [72,78] : VALUES : -15.176 <> -15.176
DIFFER : VARIABLE : precipitation_advection_y_velocity : POSITION : [72,82] : VALUES : -15.6904 <> -15.6904
...
=============================================
data/nowcast-optical-flow-from-winds/kgo_30min.nc
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : time : 9 <> 8 : VALUES : "gregorian" : "standard"
=============================================
data/nowcast-optical-flow/basic/kgo.nc
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : VARIABLE : precipitation_advection_x_velocity : POSITION : [0,23] : VALUES : 2.39487 <> 2.39487
DIFFER : VARIABLE : precipitation_advection_x_velocity : POSITION : [0,25] : VALUES : 2.38862 <> 2.38862
DIFFER : VARIABLE : precipitation_advection_x_velocity : POSITION : [0,27] : VALUES : 2.37958 <> 2.37958
DIFFER : VARIABLE : precipitation_advection_x_velocity : POSITION : [0,28] : VALUES : 2.37402 <> 2.37402
DIFFER : VARIABLE : precipitation_advection_x_velocity : POSITION : [0,29] : VALUES : 2.36777 <> 2.36777
DIFFER : VARIABLE : precipitation_advection_y_velocity : POSITION : [0,0] : VALUES : 0.144465 <> 0.144465
DIFFER : VARIABLE : precipitation_advection_y_velocity : POSITION : [0,1] : VALUES : 0.144465 <> 0.144465
DIFFER : VARIABLE : precipitation_advection_y_velocity : POSITION : [0,2] : VALUES : 0.144465 <> 0.144465
DIFFER : VARIABLE : precipitation_advection_y_velocity : POSITION : [0,3] : VALUES : 0.144465 <> 0.144465
DIFFER : VARIABLE : precipitation_advection_y_velocity : POSITION : [0,4] : VALUES : 0.144465 <> 0.144465
...
=============================================
data/nowcast-optical-flow/remasked/kgo.nc
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : VARIABLE : precipitation_advection_x_velocity : POSITION : [0,0] : VALUES : 0.193287 <> 0.193287
DIFFER : VARIABLE : precipitation_advection_x_velocity : POSITION : [0,1] : VALUES : 0.193287 <> 0.193287
DIFFER : VARIABLE : precipitation_advection_x_velocity : POSITION : [0,2] : VALUES : 0.193287 <> 0.193287
DIFFER : VARIABLE : precipitation_advection_x_velocity : POSITION : [0,3] : VALUES : 0.193287 <> 0.193287
DIFFER : VARIABLE : precipitation_advection_x_velocity : POSITION : [0,4] : VALUES : 0.193287 <> 0.193287
DIFFER : VARIABLE : precipitation_advection_y_velocity : POSITION : [0,0] : VALUES : 0.645462 <> 0.645462
DIFFER : VARIABLE : precipitation_advection_y_velocity : POSITION : [0,1] : VALUES : 0.645462 <> 0.645462
DIFFER : VARIABLE : precipitation_advection_y_velocity : POSITION : [0,2] : VALUES : 0.645462 <> 0.645462
DIFFER : VARIABLE : precipitation_advection_y_velocity : POSITION : [0,3] : VALUES : 0.645462 <> 0.645462
DIFFER : VARIABLE : precipitation_advection_y_velocity : POSITION : [0,4] : VALUES : 0.645462 <> 0.645462
...
=============================================
data/orographic_enhancement/basic/kgo_hi_res.nc
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : forecast_reference_time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : VARIABLE : orographic_enhancement : POSITION : [0,21] : VALUES : 1.28292e-07 <> 0
DIFFER : VARIABLE : orographic_enhancement : POSITION : [0,22] : VALUES : 1.74558e-07 <> 1.53845e-07
DIFFER : VARIABLE : orographic_enhancement : POSITION : [0,23] : VALUES : 1.54297e-07 <> 1.62742e-07
DIFFER : VARIABLE : orographic_enhancement : POSITION : [0,24] : VALUES : 5.40521e-08 <> 9.9406e-08
DIFFER : VARIABLE : orographic_enhancement : POSITION : [0,25] : VALUES : 7.06335e-09 <> 7.06335e-09
DIFFER : VARIABLE : projection_y_coordinate_bnds : POSITION : [0,0] : VALUES : 722000 <> 721000
DIFFER : VARIABLE : projection_y_coordinate_bnds : POSITION : [0,1] : VALUES : 721000 <> 722000
DIFFER : VARIABLE : projection_y_coordinate_bnds : POSITION : [1,0] : VALUES : 723000 <> 722000
DIFFER : VARIABLE : projection_y_coordinate_bnds : POSITION : [1,1] : VALUES : 722000 <> 723000
DIFFER : VARIABLE : projection_y_coordinate_bnds : POSITION : [2,0] : VALUES : 724000 <> 723000
...
=============================================
data/orographic_enhancement/boundary_height/kgo_hi_res.nc
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : forecast_reference_time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : VARIABLE : orographic_enhancement : POSITION : [0,21] : VALUES : 1.45696e-07 <> 0
DIFFER : VARIABLE : orographic_enhancement : POSITION : [0,22] : VALUES : 1.96954e-07 <> 1.73994e-07
DIFFER : VARIABLE : orographic_enhancement : POSITION : [0,23] : VALUES : 1.64332e-07 <> 1.73822e-07
DIFFER : VARIABLE : orographic_enhancement : POSITION : [0,24] : VALUES : 5.11043e-08 <> 1.02181e-07
DIFFER : VARIABLE : orographic_enhancement : POSITION : [0,25] : VALUES : 3.2436e-09 <> 3.2436e-09
DIFFER : VARIABLE : projection_y_coordinate_bnds : POSITION : [0,0] : VALUES : 722000 <> 721000
DIFFER : VARIABLE : projection_y_coordinate_bnds : POSITION : [0,1] : VALUES : 721000 <> 722000
DIFFER : VARIABLE : projection_y_coordinate_bnds : POSITION : [1,0] : VALUES : 723000 <> 722000
DIFFER : VARIABLE : projection_y_coordinate_bnds : POSITION : [1,1] : VALUES : 722000 <> 723000
DIFFER : VARIABLE : projection_y_coordinate_bnds : POSITION : [2,0] : VALUES : 724000 <> 723000
...
=============================================
data/phase-change-level/with_id_attr/hail_rain_kgo.nc
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : forecast_reference_time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : VARIABLE : altitude_of_rain_from_hail_falling_level : POSITION : [0,6] : VALUES : -453.09 <> -453.09
DIFFER : VARIABLE : altitude_of_rain_from_hail_falling_level : POSITION : [0,10] : VALUES : -455.98 <> -455.98
DIFFER : VARIABLE : altitude_of_rain_from_hail_falling_level : POSITION : [0,24] : VALUES : -442.509 <> -442.509
DIFFER : VARIABLE : altitude_of_rain_from_hail_falling_level : POSITION : [0,54] : VALUES : -525.414 <> -525.414
DIFFER : VARIABLE : altitude_of_rain_from_hail_falling_level : POSITION : [0,73] : VALUES : -563.804 <> -563.804
...
=============================================
data/phase-change-level/with_id_attr/sleet_rain_kgo.nc
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : forecast_reference_time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : VARIABLE : altitude_of_rain_falling_level : POSITION : [1,44] : VALUES : 145.11 <> 140.756
DIFFER : VARIABLE : altitude_of_rain_falling_level : POSITION : [3,48] : VALUES : 133.278 <> 140.644
DIFFER : VARIABLE : altitude_of_rain_falling_level : POSITION : [8,58] : VALUES : 122.113 <> 129.479
DIFFER : VARIABLE : altitude_of_rain_falling_level : POSITION : [10,62] : VALUES : 125.013 <> 117.647
DIFFER : VARIABLE : altitude_of_rain_falling_level : POSITION : [11,64] : VALUES : 115.414 <> 122.78
...
=============================================
data/phase-change-level/with_id_attr/sleet_rain_unfilled_kgo.nc
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : forecast_reference_time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : time : 9 <> 8 : VALUES : "gregorian" : "standard"
=============================================
data/phase-change-level/with_id_attr/snow_sleet_kgo.nc
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : forecast_reference_time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : VARIABLE : altitude_of_snow_falling_level : POSITION : [4,52] : VALUES : 184.06 <> 183.802
DIFFER : VARIABLE : altitude_of_snow_falling_level : POSITION : [7,58] : VALUES : 175.152 <> 171.597
DIFFER : VARIABLE : altitude_of_snow_falling_level : POSITION : [12,68] : VALUES : 150.204 <> 160.735
DIFFER : VARIABLE : altitude_of_snow_falling_level : POSITION : [15,75] : VALUES : 175.081 <> 149.202
DIFFER : VARIABLE : altitude_of_snow_falling_level : POSITION : [16,68] : VALUES : 170.767 <> 170.767
...
=============================================
data/phase-change-level/without_id_attr/hail_rain_kgo.nc
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : forecast_reference_time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : VARIABLE : altitude_of_rain_from_hail_falling_level : POSITION : [0,6] : VALUES : -453.09 <> -453.09
DIFFER : VARIABLE : altitude_of_rain_from_hail_falling_level : POSITION : [0,10] : VALUES : -455.98 <> -455.98
DIFFER : VARIABLE : altitude_of_rain_from_hail_falling_level : POSITION : [0,24] : VALUES : -442.509 <> -442.509
DIFFER : VARIABLE : altitude_of_rain_from_hail_falling_level : POSITION : [0,54] : VALUES : -525.414 <> -525.414
DIFFER : VARIABLE : altitude_of_rain_from_hail_falling_level : POSITION : [0,73] : VALUES : -563.804 <> -563.804
...
=============================================
data/phase-change-level/without_id_attr/sleet_rain_kgo.nc
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : forecast_reference_time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : VARIABLE : altitude_of_rain_falling_level : POSITION : [1,44] : VALUES : 145.11 <> 140.756
DIFFER : VARIABLE : altitude_of_rain_falling_level : POSITION : [3,48] : VALUES : 133.278 <> 140.644
DIFFER : VARIABLE : altitude_of_rain_falling_level : POSITION : [8,58] : VALUES : 122.113 <> 129.479
DIFFER : VARIABLE : altitude_of_rain_falling_level : POSITION : [10,62] : VALUES : 125.013 <> 117.647
DIFFER : VARIABLE : altitude_of_rain_falling_level : POSITION : [11,64] : VALUES : 115.414 <> 122.78
...
=============================================
data/phase-change-level/without_id_attr/sleet_rain_unfilled_kgo.nc
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : forecast_reference_time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : time : 9 <> 8 : VALUES : "gregorian" : "standard"
=============================================
data/phase-change-level/without_id_attr/snow_sleet_kgo.nc
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : forecast_reference_time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : VARIABLE : altitude_of_snow_falling_level : POSITION : [4,52] : VALUES : 184.06 <> 183.802
DIFFER : VARIABLE : altitude_of_snow_falling_level : POSITION : [7,58] : VALUES : 175.152 <> 171.597
DIFFER : VARIABLE : altitude_of_snow_falling_level : POSITION : [12,68] : VALUES : 150.204 <> 160.735
DIFFER : VARIABLE : altitude_of_snow_falling_level : POSITION : [15,75] : VALUES : 175.081 <> 149.202
DIFFER : VARIABLE : altitude_of_snow_falling_level : POSITION : [16,68] : VALUES : 170.767 <> 170.767
...
=============================================
data/phase-mask/rain/kgo.nc
DIFFER : VALUES OF GLOBAL ATTRIBUTE : Conventions : CF-1.5 <> CF-1.7
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : forecast_reference_time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : time : 9 <> 8 : VALUES : "gregorian" : "standard"
=============================================
data/phase-mask/sleet/kgo.nc
DIFFER : VALUES OF GLOBAL ATTRIBUTE : Conventions : CF-1.5 <> CF-1.7
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : forecast_reference_time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : time : 9 <> 8 : VALUES : "gregorian" : "standard"
=============================================
data/phase-mask/snow/kgo.nc
DIFFER : VALUES OF GLOBAL ATTRIBUTE : Conventions : CF-1.5 <> CF-1.7
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : forecast_reference_time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : time : 9 <> 8 : VALUES : "gregorian" : "standard"
=============================================
data/phase-probability/equalarea/hail_kgo_deterministic.nc
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : forecast_reference_time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : time : 9 <> 8 : VALUES : "gregorian" : "standard"
=============================================
data/phase-probability/equalarea/hail_kgo_percentiles.nc
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : forecast_reference_time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : time : 9 <> 8 : VALUES : "gregorian" : "standard"
=============================================
data/phase-probability/equalarea/rain_kgo_deterministic.nc
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : forecast_reference_time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : time : 9 <> 8 : VALUES : "gregorian" : "standard"
=============================================
data/phase-probability/equalarea/rain_kgo_percentiles.nc
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : forecast_reference_time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : time : 9 <> 8 : VALUES : "gregorian" : "standard"
=============================================
data/phase-probability/equalarea/snow_kgo_deterministic.nc
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : forecast_reference_time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : time : 9 <> 8 : VALUES : "gregorian" : "standard"
=============================================
data/phase-probability/equalarea/snow_kgo_percentiles.nc
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : forecast_reference_time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : time : 9 <> 8 : VALUES : "gregorian" : "standard"
=============================================
data/phase-probability/latlon/hail_kgo_deterministic.nc
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : forecast_reference_time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : time : 9 <> 8 : VALUES : "gregorian" : "standard"
=============================================
data/phase-probability/latlon/rain_kgo_deterministic.nc
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : forecast_reference_time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : time : 9 <> 8 : VALUES : "gregorian" : "standard"
=============================================
data/phase-probability/latlon/snow_kgo_deterministic.nc
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : forecast_reference_time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : time : 9 <> 8 : VALUES : "gregorian" : "standard"
=============================================
data/phase-probability/spot/hail_kgo_deterministic.nc
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : forecast_reference_time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : time : 9 <> 8 : VALUES : "gregorian" : "standard"
=============================================
data/phase-probability/spot/hail_kgo_percentiles.nc
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : forecast_reference_time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : time : 9 <> 8 : VALUES : "gregorian" : "standard"
=============================================
data/phase-probability/spot/rain_kgo_deterministic.nc
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : forecast_reference_time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : time : 9 <> 8 : VALUES : "gregorian" : "standard"
=============================================
data/phase-probability/spot/rain_kgo_percentiles.nc
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : forecast_reference_time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : time : 9 <> 8 : VALUES : "gregorian" : "standard"
=============================================
data/phase-probability/spot/snow_kgo_deterministic.nc
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : forecast_reference_time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : time : 9 <> 8 : VALUES : "gregorian" : "standard"
=============================================
data/phase-probability/spot/snow_kgo_percentiles.nc
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : forecast_reference_time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : time : 9 <> 8 : VALUES : "gregorian" : "standard"
=============================================
data/precipitation_duration/renamed/kgo.nc
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : forecast_reference_time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : time : 9 <> 8 : VALUES : "gregorian" : "standard"
=============================================
data/precipitation_duration/spot_data/kgo.nc
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : forecast_reference_time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : time : 9 <> 8 : VALUES : "gregorian" : "standard"
=============================================
data/precipitation_duration/standard_names/kgo_acc_0.10_rate_4.0.nc
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : forecast_reference_time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : time : 9 <> 8 : VALUES : "gregorian" : "standard"
=============================================
data/precipitation_duration/standard_names/kgo_acc_1.00_rate_4.0.nc
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : forecast_reference_time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : time : 9 <> 8 : VALUES : "gregorian" : "standard"
=============================================
data/precipitation_duration/standard_names/kgo_multi_threshold.nc
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : forecast_reference_time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : time : 9 <> 8 : VALUES : "gregorian" : "standard"
=============================================
data/precipitation_duration/standard_names/kgo_short_period.nc
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : forecast_reference_time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : time : 9 <> 8 : VALUES : "gregorian" : "standard"
=============================================
data/recursive-filter/kgo_basic.nc
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : forecast_reference_time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : time : 9 <> 8 : VALUES : "gregorian" : "standard"
=============================================
data/recursive-filter/kgo_internal_mask_with_re_mask.nc
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : forecast_reference_time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : time : 9 <> 8 : VALUES : "gregorian" : "standard"
=============================================
data/recursive-filter/kgo_variable_internal_mask_with_re_mask.nc
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : forecast_reference_time : 9 <> 8 : VALUES : "gregorian" : "standard"
DIFFER : LENGTHS : ATTRIBUTE : calendar : VARIABLE : time : 9 <> 8 : VALUES : "gregorian" : "standard"

@bayliffe
Copy link
Contributor Author

bayliffe commented Jul 8, 2025

data/nowcast-optical-flow-from-winds/kgo_15min.nc

Precision level differences, ignorable:

Y-direction max fractional change: 1.1917499165292611e-07
X-direction max fractional change: 9.179112936408274e-08

data/nowcast-optical-flow/basic/kgo.nc & data/nowcast-optical-flow/remasked/kgo.nc

np.linalg.inv method makes extremely small changes; same inputs, very slightly different outputs.
These get multiplied up to give differences of up to 5E-7.

e.g.

Inputs: deriv_xy 0.000394888301671017 0.008808541332823024 float64
Old output: velocity -3.4059884931540103 0.5292365813135453 float64
New output: velocity -3.4059884931540187 0.5292365813135447 float64

data/orographic_enhancement/basic/kgo_hi_res.nc

The original data has the bounds defined in the reverse direction to the direction of the y coordinate.
Reading the coordinate bounds with the netCDF4 library (to be iris independent) the old kgo gives:

masked_array(
  data=[[722000., 721000.],
        [723000., 722000.],
        [724000., 723000.],
        [725000., 724000.],
        [726000., 725000.],

The new kgo gives:

masked_array(
  data=[[721000., 722000.],
        [722000., 723000.],
        [723000., 724000.],
        [724000., 725000.],
        [725000., 726000.],

This is a result of use of the sort_coord_in_cube function in the orographic enhancement plugin (https://github.com/metoppv/improver/blob/8c959471eb677cec4d621af98bbb58d4c36df46e/improver/utilities/cube_manipulation.py#L465). The hi res topography cube passed in has the y-coordinate defined in descending order. The orographic enhancement plugin sorts the x and y coordinates into ascending order. This moves around the associated bounds, but doesn't reorder the bounds in each pair to also be ascending. So:

1.5 [2, 1]
0.5 [1, 0]

becomes

0.5 [1, 0]
1.5 [2, 1]

The change described here SciTools/iris#4466 which allows bounds to be in reverse order and puts in associated checks is probably responsible for the bounds pairs now being reversed appropriately.

This change in the bounds actually changes the results around the boundary of the domain, e.g.

DIFFER : VARIABLE : orographic_enhancement : POSITION : [0,21] : VALUES : 1.28292e-07 <> 0
DIFFER : VARIABLE : orographic_enhancement : POSITION : [0,22] : VALUES : 1.74558e-07 <> 1.53845e-07
DIFFER : VARIABLE : orographic_enhancement : POSITION : [0,23] : VALUES : 1.54297e-07 <> 1.62742e-07
DIFFER : VARIABLE : orographic_enhancement : POSITION : [0,24] : VALUES : 5.40521e-08 <> 9.9406e-08
DIFFER : VARIABLE : orographic_enhancement : POSITION : [0,25] : VALUES : 7.06335e-09 <> 7.06335e-09

Some of these changes are substantial (e.g. factor of 2, or infinite when changing to/from 0). I've not attempted to pin down why but I presume some coordinate matching has changed due to the differing bounds. The bounds should be defined as they are in the new result, meaning I believe it was previously wrong. I have tested that this is the source of the differences by modifying the master version of the code as below and running the test with the old environment.

        for axis in ["x", "y"]:
            topography = sort_coord_in_cube(topography, topography.coord(axis=axis))
            abounds = topography.coord(axis=axis).bounds
            topography.coord(axis=axis).bounds = np.flip(abounds, axis=1)

With the bounds reordered the output from the old code and environment is identical to the result with the environment upgrade branch of improver and the new environment. The difference we see between the old KGO and the new KGO are plotted below.

orog_enhance_change

data/orographic_enhancement/boundary_height/kgo_hi_res.nc

I'm assuming the issues are all the same as above.

@bayliffe
Copy link
Contributor Author

bayliffe commented Jul 8, 2025

Phase change levels

data/phase-change-level/without_id_attr/sleet_rain_unfilled_kgo.nc

Only calendar changes

The rest

data/phase-change-level/with_id_attr/hail_rain_kgo.nc
data/phase-change-level/with_id_attr/sleet_rain_kgo.nc
data/phase-change-level/with_id_attr/sleet_rain_unfilled_kgo.nc
data/phase-change-level/with_id_attr/snow_sleet_kgo.nc
data/phase-change-level/without_id_attr/hail_rain_kgo.nc
data/phase-change-level/without_id_attr/sleet_rain_kgo.nc
data/phase-change-level/without_id_attr/snow_sleet_kgo.nc

Values change at the call to griddata using the nearest method. See @gavinevans's comment about this:
https://github.com/metoppv/mo-blue-team/issues/940#issuecomment-3032988535

The returned type is also escalated from float32 to float64 which it was not previously, but this is inserted into a float32 array and so reset upon return from the interpolate_missing_data method.

We can see the structured differences that result from neighbour selection differences, e.g.

data/phase-change-level/with_id_attr/hail_rain_kgo.nc

regrid_diff

@bayliffe
Copy link
Contributor Author

bayliffe commented Jul 8, 2025

All of the remaining differences are just changes to the calendars.

@bayliffe bayliffe marked this pull request as ready for review July 8, 2025 11:39
@bayliffe bayliffe changed the base branch from master to environment_upgrade_feature_branch July 8, 2025 11:53
Copy link
Contributor

@mspelman07 mspelman07 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm happy that you've addressed and explained all the changes present in these tests

@mspelman07 mspelman07 merged commit ebdbae0 into environment_upgrade_feature_branch Jul 9, 2025
bayliffe added a commit that referenced this pull request Jul 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants