cftime again

[rcomer]

It looks like new versions of cftime have been released in the last week. Travis tests are failing again with the same errors we were getting before #3215, which pinned cftime to not 1.0.2.1. For example of failing tests, see #3238.

[cpelley]

This effects a number of our users.
To that end, #3215 suggests that it is pinning cftime to a version less than 1.0.2.1 but the change itself looks like to only specifically precluded version 1.0.2.1

Currently, doing a conda install of iris via conda-forge leads to problems for our users due to cftime.

I'm going to propose that this be dealt with in two stages (the first, an immediate fix giving breathing room for the second):

• Stage:1 Return the pre-existing behaviour by pinning cftime to <1.0.2.1
• Stage:2 Look at iris supporting cftime >= 1.0.2.1
  • To be clear here. Any change of behaviour here might have significant impact to users so needs careful consideration. For example, a number of ancillaries in the wild have time information populated with 0. For better or for worse, common/historical usage is of importance here.

[bjlittle]

I agree with the approach suggested by @cpelley.

It's a bit of a weird situation, as I managed to get a fix merged into cftime that helps resolve issues with iris time partial date constraints, but that is only available in cftime 1.0.3.

cftime has made several changes recently, and the last fix we need to put in place is dealing with 0 based dates - which technically are indeed illegal, the reality is that there are quite a lot of them out there, so we need to deal with that in iris some how if we want to use the latest cftime

[pp-mo]

Re:

Stage:2 Look at iris supporting cftime >= 1.0.2.1 ... ancillaries in the wild ... common/historical usage

#3245 mentions this.
@DPeterK initially proposed to adjust the iris saving
But we then agreed it is better to fix loading (too?)

This is now awaiting a more suitable solution.
#3248 refers

At bottom, we should really be loading data with '0 times' with no time coord.
We can then save such with '0 times', too.
This makes good sense, but it does involve behaviour change.

[cpelley]

• Additional context: this issue doesn't appear to effect the usecase from Add 'month' coord instead of 'time' in pp_rules when lbyr=0 and lbtim.ib==2 #1926 (this is where there is a year 0 but non-zero month, day, ...). Identified by the following tests passing:
  iris.tests.unit.fileformats.pp_load_rules.test__all_other_rules.TestLBTIMx2x_ZeroYears
iris.tests.unit.fileformats.pp_load_rules.test__convert_time_coords.TestLBTIMx2x_ZeroYear

[pp-mo]

For now, I'm proposing to "fix" this by wrapping the usage in iris.fileformats.pp_load_rules to reproduce the "old-style" results for a date-to-num conversion.
#3259

That's really just a patch "for now" : it doesn't address any of the better ideas raised here,
But if problems are confined to PP/FF conversion, that may be good enough + we can close this down ??

[pp-mo]

#3259 ... a patch "for now" ... may be good enough ...
we can close this down ??

Opinion, anyone ?

[rcomer]

Opinion, anyone ?

When I opened this, I was mostly just worried about getting the Travis tests passing. So from that POV, we're good. But the coversation has become more wide-ranging

if problems are confined to PP/FF

I don't think they are. Try this:

import iris

time_coord = iris.coords.DimCoord(range(5), units='days since 2001-05-01',
                                  standard_name='time')

# Add as aux coord
cube = iris.cube.Cube(range(5))
cube.add_aux_coord(time_coord, 0)

# Save and then load, attempt to print time coord.
print(cube.coord('time'))
iris.save(cube, 'test_cube.nc')
temp_cube = iris.load_cube('test_cube.nc')
print(temp_cube.coord('time'))

This gives an exception with Iris 2.2 and cftime 1.0.0b1. Weirdly, if time_coord is added as a dimcoord, all is well. Sometimes time is 2D, so adding it as a dimcoord may not be possible.

[pp-mo]

Well I think the tests are now ok, so I'm going to close this.

The previous comment seem to me to be an unrelated issue, as it doesn't actually involve "zero-datetimes". I had hope to work out what it is about, but it seems to be a real puzzler, so I'm going to create another issue for it now... #3402