Proposal: Python / PostgreSQL / Etc. Version Deprecation Policy

[callahad]

Synapse currently claims to support Python 3.5 (EOL Sept. 2020) and Postgres 9.5 (EOL Feb. 2021).

We need to plan for how we will remove support for these platform dependencies in a responsible manner.

Motivation

I'm afraid we're boxing ourselves into a corner where our dependencies might be unsupported on those platform versions, and if a security vulnerability were to arise, we wouldn't be in a position to address it without maintaining an untenable number of forks / backports or forcing all of our customers into an emergency upgrade of their infrastructure.

Articulating a support lifecycle allows our customers to make reasonable plans for managing their infrastructure and allows us to know when we can take advantage of newer platform features.

Proposal

1. We follow upstream support lifecycles.
   When a version of a language, database, or Linux distribution reaches End of Life, we also withdraw our support for that version.

2. There is no guaranteed grace period.
   We will not intentionally break compatibility for at least one month after an EOL date, but we explicitly reserve the right to do so should security or other concerns arise in an unmaintained version of a dependency.

3. This comes into effect March 2021.
   We'll announce with the next Synapse release. Synapse will be allowed to break Py3.5, Pg9.5, and Ubuntu 16.04 starting with our first release in March 2021.

Additional Context

• Element is happy with this policy, including folks who liaise with large EMS and Professional Services customers.
• Ubuntu Xenial (16.04 LTS) ships Py3.5 and Pg9.5; it goes EOL Apr. 2021. This seems to align closely enough with the proposed timeline.
• Platform support lifecycles are relatively long. Python, Postgres, and Ubuntu tend to have annual or biannual releases with five-year support windows. This ensures multiple years of overlap between supported releases; ample time for organizations to validate and deploy new platform versions.
• Folks who are really stuck on old distros can use our Docker images instead. Or make their own packages.

Questions

• Is a one month, best-effort grace period sufficient?
• Do we need to explicitly enumerate our supported Linux distros somewhere given that we do build our own Debian / Ubuntu packages, or can we some how get away without doing that?
• Does the roll-out (announce next release, implement in March) sound right?

[clokep]

Cherry-picking a couple of things:

Ubuntu Xenial (16.04 LTS) ships Py3.5 and Pg9.5; it goes EOL Feb. 2021. This seems to align nicely with the proposed timeline.

Feb. 2021 is the end of standard support, extended security maintenance goes through April 2024. (Also it seems to be April 2021, not Feb. 2021?)

We will not intentionally break compatibility for at least one month after an EOL date, but we explicitly reserve the right to do so should security or other concerns arise in an unmaintained version of a dependency.

Is a one month, best-effort grace period sufficient?

I wonder if this should be a number of releases, not time? (Or some combination?) If EOL was Jan. 1 for some dependency, but we don't release until Feb. 1, does that mean we can drop support for it?

Do we need to explicitly enumerate our supported Linux distros somewhere given that we do build our own Debian / Ubuntu packages, or can we some how get away without doing that?

If we're supporting the supported Debians / Ubuntus I'm not sure this is necessary.

[callahad]

[Ubuntu 16.04's] extended security maintenance goes through April 2024

When we're at the point of a distro entering a paid ESM period... Synapse just isn't stable / static enough to be a core part of that ecosystem. Docker itself is likely to be covered by the ESM contract, which lets us keep a supported avenue for running Synapse on those platforms, just not as directly.

(Also it seems to be April 2021, not Feb. 2021?)

You would think I would've noticed that, given the "04" in "16.04". Fixed 🙂

I wonder if this should be a number of releases, not time? (Or some combination?) If EOL was Jan. 1 for some dependency, but we don't release until Feb. 1, does that mean we can drop support for it?

I'd strongly argue that yes, we can and should drop support on a fixed timescale after EOL. Mainly because we're not pushing anyone to anything near the cutting edge. Rather, we're sweeping up the very last trailing versions. By the time we drop Python 3.6, people will have had five years to find a way to move to at least 3.7. Accommodation beyond that point is unlikely to yield significant benefit. 😉

[clokep]

I wonder if this should be a number of releases, not time? (Or some combination?) If EOL was Jan. 1 for some dependency, but we don't release until Feb. 1, does that mean we can drop support for it?

I'd strongly argue that yes, we can and should drop support on a fixed timescale after EOL. Mainly because we're not pushing anyone to anything near the cutting edge. Rather, we're sweeping up the very last trailing versions. By the time we drop Python 3.6, people will have had five years to find a way to move to at least 3.7. Accommodation beyond that point is unlikely to yield significant benefit. 😉

I guess I'm confused at why we don't just drop it immediately then. The only benefit is if we do a release within the 1 month time frame. Why not just say we support it for one more release past EOL, then drop it.

[callahad]

I'm fond of a time-based grace period because it's predictable (Python 3.6's EOL is a known date), while our release timing and cadence could vary, but I could be persuaded either way. I'd appreciate other folks weighing in.

[callahad]

Potentially lobbing a grenade into the discussion... maybe we shouldn't care about Linux distro lifetimes at all? We depend on Python and Posgres, not a specific operating system.

I'm almost inclined to say that once our direct dependencies are EOL, then we move on. If that means we no longer work on a distro that's otherwise supported by its vendor, then that's unfortunate, but it's not our place to solve it. We have minimum platform requirements, and it's each admin's responsibility to provide a runtime environment which fulfills those requirements.

Otherwise we get into a weird place of having to contort our work to fit with the schedules of an arbitrary collection of distros. This potentially puts us in a place where we're stuck supporting software that the rest of the ecosystem has long since left behind. And the more distros we support, the more challenging that alignment becomes. (Plus, which distros do we anoint? If Ubuntu, then also Debian? But what about RHEL? Or SuSE? Maybe even FreeBSD?)

It's important that our minimum requirements are reasonable, and I think they are. If we follow upstream support lifetimes, then we'll never require a language runtime or database version that's anything less than several years old.

[callahad]

As mentioned by @clokep, similar questions are being discussed by Django right now. The initial proposal over there is roughly in line with ours:

ideally, we'd follow Python's support versions policy. (The flip side being I think we should probably be actively dropping support for versions of Python as they go EOL, even if between [Django] LTS versions.)

[callahad]

TODO: Announce with 1.25.0's changelog, for later implementation.