RFC: Python and Numpy version support and release schedule

[effigies]

Summary

This issue exists to consolidate discussion on support schedules for dependencies, in particular Python and Numpy, as these form the basis of the scientific Python ecosystem and are generally the most painful to upgrade. It additionally provides the canonical plan of release dates and version support matrix.

As release dates and support schedules get decided and modified, this schedule should be updated.

Schedule

Milestone/Tag	Date	Python	numpy	Notes
1.2.2	2019.09.07	2.7, 3.4+	1.9+	Off-schedule release to support downstream projeects
1.2.3	2019.09.23	2.7, 3.4+	1.9+	Final Python 3.4 release
1.3.0	2019.11.11	2.7, 3.5+	1.12+	Final Python 2.7 series, begin 1.3.x bug-fix only branch; Drop 2.7 on master; import workflows from niflows
1.3.1	2019.11.12	2.7, 3.5+	1.12+	1.3.0 + backported bug fixes only
1.4.0	2019.12.16	3.5+	1.12+	All new features post-1.3.0
1.3.2	2020.01.06	2.7, 3.5+	1.12+	1.3.1 + backported bug fixes only; further 1.3.x releases only as needed
1.4.1	2020.01.27	3.5+	1.12+
1.5.0	2020.02.24	3.5+	1.13+	Move nipype1 examples to niflows
...
2.0.0	TBD	3.7+	TBD	pydra-based

Support windows

Numpy has published NEP 29, a recommended deprecation policy for scientific software. While Nipype does not make heavy use of Numpy, we do interact with a lot of software that does, and so it seems wise to pay attention to this plan.

In nibabel (nipy/nibabel#803) it appears we're likely to follow a plan of NEP + 1 year in our schedule for dropping support for Python and Numpy, which is the approximate difference between our ad hoc support plan and Numpy's (relatively) aggressive plan. I've followed that schedule in the numpy releases above.

Related threads

• MNT: Drop Python 2 support #2654

[satra]

@effigies - thank you for putting this together. looks like an excellent plan.

[oesteban]

Judging by #3046, would it make sense to drop Python 3.5 soon too?

Milestone/Tag	Date	Python	numpy	Notes
1.2.2	2019.09.07	2.7, 3.4+	1.9+	Off-schedule release to support downstream projeects
1.2.3	2019.09.23	2.7, 3.4+	1.9+	Final Python 3.4 and 3.5 releases
1.3.0	2019.10.28	2.7, 3.6+	1.12+	Final Python 2.7 series, begin 1.3.x bug-fix only branch; Drop 2.7 on master; import workflows from niflows
1.3.1	2019.11.25	2.7, 3.6+	1.12+	1.3.0 + backported bug fixes only; further 1.3.x releases only as needed
1.4.0	2019.12.16	3.6+	1.12+	All new features post-1.3.0
1.4.1	2020.01.27	3.6+	1.12+
1.5.0	2020.02.24	3.6+	1.13+	Move nipype1 examples to niflows
...
2.0.0	TBD	3.7+	TBD	pydra-based

[effigies]

I would suggest having a principled and predictable deprecation schedule, whatever it may be.

The initial suggestion follows nibabel's schedule of NEP 29 + 1 year, but nipype is not bound to be as conservative as that. If you want to follow NEP 29 with no lag, then we should drop numpy 1.13 and Python 3.5 as of 1.3.0, and we'll drop Python 3.6 next summer.

I don't have a strong opinion about what route nipype follows, but it does feel early to me to drop 3.5, though that may be just because most projects are only just now dropping 3.4. It also seems odd to me to drop 3.5 before 2.7. Would a useful compromise be to keep on with the current plan for 1.3.x and then adopt NEP 29 for nipype 1.4+?

Whatever our schedule, I think #3046 is a good warning against monkey-patching external modules, and that we should be careful about using new idioms that require shims or monkey-patches to work throughout the support window.

[effigies]

@oesteban @satra Do you think we should push a mid-month 1.2.4 that fixes these late issues (#3046, #3049, #3050, #3047/#3061), or are we going to go ahead with 1.3.0?

If 1.3.0, I wouldn't mind removing 3.4 from the test suite just to keep things moving a little faster, but I've been holding off just in case.

[satra]

i think this is not just a minor adjustment, so i would be fine with 1.3.0 and removing 3.4, and calling 1.2.3 a bad version

[effigies]

Any opposition to a 1.3.0-rc1 this week? These issues are causing a lot of pain in CI and with user support.

[satra]

nope.

[effigies]

A couple notes on release candidates:

• In order for conda-forge to work smoothly, the release branches need to be named rel/X.Y.ZrcN, not rel/X.Y.Z-rcN, and the tag should be named X.Y.ZrcN as well. These normalize to the same PEP440 version string, with or without the dash, so we can continue using the dash in our version strings, if we want.
• For dev versions between an RC and final release, I'm using X.Y.Z-rcN.post-dev:

>>> from packaging.version import Version
>>> Version('1.3.0-dev') < Version('1.3.0-rc1') < Version('1.3.0-rc1.post-dev') < Version('1.3.0-rc2') < Version('1.3.0')                                                                                                                                                        
True