Meeting: Release plan bikeshed

[rvagg]

Building on discussions in many previous issues, the status of which is best summarised in #2215, we're trying to get to some closure here, particularly to also get alignment with the LTS plan.

In this week's LTS meeting and today's (yesterday's?) TSC meeting, we're starting to get a stronger consensus around a model that has been outlined by @trevnorris about how we use branches and master in particular. Most of us simply have fatigue over this and just want to land on something practical.

Remaining details that are still in flux are mostly around how to deal with ongoing V8 upgrades in master and other breaking changes and how this impacts canary/next/whatever builds and people relying on them and mostly how that impacts on contributors coming to the project not familiar with how we have it set up. Another topic for discussion is the process by which those changes are propagated around to other branches. Do we have a strict policy of landing all changes on master that fit there or can we be more flexible about where things land and then cherry-pick? Do we have any use for merges with this plan or is it all cherry-picking? (I'd ask about rebasing but I fear the reaction to that). And what mechanisms can we employ, human and technological, to ensure that changes propagate where they should and important commits don't get lost in the master firehose.

SO, with that, I'm proposing yet another meeting, a separate one, for those interested in trying to hash out enough of the details that we can present a plan that's complete enough to be able to vote on.

Because I really don't want another 6am I'm using 1 hour later than our regular slot, ~1 day 7 hours the time of this post.

9:00 PM Friday, 24 July 2015 UTC: http://www.timeanddate.com/worldclock/fixedtime.html?msg=Release%20plan%20bikeshed+2015-07-24&iso=20150724T21

Hangout on Air for participants: https://plus.google.com/hangouts/_/hoaevent/AP36tYdv26U8GY0wS7lCe9wMCCORdt4c7-QzcKxnko5q80UDKfK51Q

YouTube feed for lurkers and watching later: http://www.youtube.com/watch?v=JwwTZIRUoHA

This is not restricted to the TSC if you think you can make a valuable contribution to this; just be aware that (a) we have a limit on the number of people that can join the HOA and (b) some of us are sick of endless discussion about this topic.

Pinging @nodejs/tsc, and particularly @trevnorris, @jasnell, @mikeal, @chrisdickinson, I hope y'all can make it.

[jasnell]

+1. Will be there
On Jul 23, 2015 6:59 AM, "Rod Vagg" [email protected] wrote:

Building on discussions in many previous issues, the status of which is
best summarised in #2215 #2215,
we're trying to get to some closure here, particularly to also get
alignment with the LTS plan.

In this week's LTS meeting and today's (yesterday's?) TSC meeting, we're
starting to get a stronger consensus around a model that has been outlined
by @trevnorris https://github.com/trevnorris about how we use branches
and master in particular. Most of us simply have fatigue over this and
just want to land on something practical.

Remaining details that are still in flux are mostly around how to deal
with ongoing V8 upgrades in master and other breaking changes and how
this impacts canary/next/whatever builds and people relying on them and
mostly how that impacts on contributors coming to the project not familiar
with how we have it set up. Another topic for discussion is the process by
which those changes are propagated around to other branches. Do we have a
strict policy of landing all changes on master that fit there or can we
be more flexible about where things land and then cherry-pick? Do we have
any use for merges with this plan or is it all cherry-picking? (I'd ask
about rebasing but I fear the reaction to that). And what mechanisms can we
employ, human and technological, to ensure that changes propagate where
they should and important commits don't get lost in the master firehose.

SO, with that, I'm proposing yet another meeting, a separate one, for
those interested in trying to hash out enough of the details that we can
present a plan that's complete enough to be able to vote on.

Because I really don't want another 6am I'm using 1 hour later than our
regular slot, ~1 day 7 hours the time of this post.

9:00 PM Friday, 24 July 2015 UTC:
http://www.timeanddate.com/worldclock/fixedtime.html?msg=Release%20plan%20bikeshed+2015-07-24&iso=20150724T21

Hangout on Air for participants:
https://plus.google.com/hangouts/_/hoaevent/AP36tYdv26U8GY0wS7lCe9wMCCORdt4c7-QzcKxnko5q80UDKfK51Q

YouTube feed for lurkers and watching later:
http://www.youtube.com/watch?v=JwwTZIRUoHA

This is not restricted to the TSC if you think you can make a valuable
contribution to this; just be aware that (a) we have a limit on the number
of people that can join the HOA and (b) some of us are sick of endless
discussion about this topic.

Pinging @nodejs/tsc https://github.com/orgs/nodejs/teams/tsc, and
particularly @trevnorris https://github.com/trevnorris, @jasnell
https://github.com/jasnell, @mikeal https://github.com/mikeal,
@chrisdickinson https://github.com/chrisdickinson, I hope y'all can
make it.

—
Reply to this email directly or view it on GitHub
#2230.

[bnoordhuis]

I'll try to make it.

[rvagg]

For further context, I have a blog post up https://medium.com/@nodesource/essential-steps-long-term-support-for-node-js-8ecf7514dbd about the current LTS plan and how it fits in with the current thinking around the release process.

I think this clarifies the model a little:

I'm not suggesting that master-as-unstable is locked in as the plan but it's currently looking like the most viable option and modelling it is much simpler than trying to do next/master flip flop.

[orangemocha]

11pm here, I won't be able to make it. I have full confidence in your ability to nail this down 😄
Looking forward to seeing the policy documented somewhere so that all collaborators know what to do....

[PinkaminaDianePie]

@rvagg only one upgrade of v8 for half of year? Or it will be upgraded with minor versions?

[mikeal]

@PinkaminaDianePie only one major release with a new v8 every year. the plan is to do regular alpha releases of master which will take v8 on a faster schedule.

[Fishrock123]

@PinkaminaDianePie

It's not possible to upgrade v8 faster, or to do it in minor versions. v8's c++ API is constantly breaking in their N.N.0.0 versions, and routinely breaks large parts our c++ and c++ bindings.

The plan:

Have a dev branch which upgrades v8 asap. There will be coinciding releases of this Branch, probably marked as "dev" or "beta". The warranty on using native (c / c++) addons if you use this branch if void, since v8 routinely breaks these in major ways.

Stable branches that cut off of dev every 6 months and bump the major. Minor and patch level fixes continue to also be landed here.

Every year cut a LTS release of of one of the two stable releases (always an even or odd major release). Essentially maintenance only.

[mikeal]

beautiful graph, can we get it under a CC license so we can stick it directly in the weekly update?

[PinkaminaDianePie]

@Fishrock123

There will be coinciding releases of this Branch, probably marked as "dev" or "beta".

this was exactly i worried about. i remember that one of the main problem of old node was very slow update of v8, and iojs switched to more frequent updates. in the last few years javascript evolves significaly and support of new features will be good out of box. even ecma changed numeration of standarts with year instead of number, so probably new standarts will be shiped every year and upgrading v8 is in high priority for some users of node.

[Fishrock123]

this was exactly i worried about. i remember that one of the main problem of old node was very slow update of v8, and iojs switched to more frequent updates. in the last few years javascript evolves significaly and support of new features will be good out of box. even ecma changed numeration of standarts with year instead of number, so probably new standarts will be shiped every year and upgrading v8 is in high priority for some users of node.

Right. Sure. I also agree with using up-to-date v8. We'll have official releases of the "dev" (master) branch that will be stable (except for native modules), and when we cut a stable it will use whichever stable v8 has been in for a few weeks to make sure native addons can be caught up for a stable release. :)

The unfortunate problem here is that v8's c++ API just breaks too much. It's not manageable for the ecosystem and it's barely manageable for core sometimes.

Bottom line is: if that's what you can't live without you'd need to hire full-time people for us who:

1. Update core and NAN.
2. Update the ecosystem for module authors.

I'm sorry it can't be better, we've already talked to the v8 team and they are in no way interested in keeping a more stable c++ API, because their real priority is chrome. :|

(You could argue "don't use v8", but then you really need to be dedicating yourself to moving us off of v8, full-time, long-term. :/)

[mikeal]

Some other important differences:

• Node.js had a default branch of the current release while master lead as the next major.
• Node.js did "alpha/beta" releases as full odd numbered releases and did them very infrequently.
• Even though master was the next major release it didn't regularly take a new v8.

The plan here is to have master be the default branch PR's land in and regularly cherry-pick to stable releases (probably weekly) and to also regularly (probably weekly) produce alpha releases of the next major off of master.

New v8 will be taken regularly in master and alpha releases produced. Community members who want new v8 features on a regular basis can pull from this alpha channel and we're actively trying to keep it stable enough for you to depend on, you're just going to need to stay away from native modules because that ecosystem won't likely be keeping up as fast as you are.

[rvagg]

@mikeal CC-BY version

https://cldup.com/lMHtbE3Nrx-3000x3000.png

[trevnorris]

Do we have a strict policy of landing all changes on master

We can't. There will be more than a few patches that only pertain to the active stable release. Especially if that release will become an LTS, we can expect patches to land right up until we no longer support it.

Do we have any use for merges with this plan or is it all cherry-picking?

I'd prefer taking the cherry-picking route to start, but have no problem re-visiting it if it's found to be impractical.

And what mechanisms can we employ, human and technological, to ensure that changes propagate

I believe we discussed using Labels on PRs to make sure they are migrated properly. It may work well to have the rules that any PR that should propagate be left open until all listed labels (e.g. for-stable) have been done.

We should make special note of the overlap of stable releases, and what this implies. That each next major will have some support overlap with the previous to make sure everyone can migrate properly (e.g. module authors).

[jasnell]

Ok, while working on writing this up, I realized that I may have misinterpreted some items. Before I get to much farther on writing this up, I just wanted to double check:

• master is unstable. commits generally land there. The overall goal is to keep master as stable as possible but every release in master is just an alpha for the next stable, so breaking changes from one master release to another are possible without bumping the major.
• every six months, major is bumped in master and a stable branch is created for the previous major. In other words, if master is currently on v3.1.3 and is being bumped to v4.0.0, the stable branch is cut off v3.1.3. Then, when master transitions to v5.0.0 six months later, a new stable is cut off v4.x.x.
• we'll have ~weekly alpha releases off master. Developers can use these but ought to expect things to break once in a while.
• there would be two stable branches each year, which will overlap by some period of time
• there will be a regular cadence (~weekly?) of stable releases off the stable branch. Developers can (and should) use these and be assured that there will be no breaking changes outside of the occasional breaking security fix. (a stable is guaranteed not to have semver-major bumps... correct?)
• of the two stables cut each year, the second will transition into the LTS. The releases cut off the LTS-bound stable are essentially the LTS release candidates.
• commits generally land in master, then are cherry picked out to each of the stable/LTS branches. Cherry-picked commits will "trickle down" through stable branches from newer branches to older (commits that land in master would cherry pick into v5.0; commits into v5.0 would cherry pick into v4.0; and so on) as opposed to the current joyent/node approach of landing in the oldest and moving forward towards master
• semver breaking critical security fixes are handled as special cases. Since the major cannot be bumped in stable or LTS branches, they will land as semver-minor bumps with a mandatory documentation requirement. This isn't ideal, but there's really no other choice.

If this looks right, I'll continue working on the write up and get that posted to the dev-policy repo later tonight.

[Fishrock123]

master is unstable. commits generally land there. The overall goal is to keep master as stable as possible but every release in master is just an alpha for the next stable, so breaking changes from one master release to another are possible without bumping the major.

I think it is better to term it something else other than "unstable". i.e. we won't call it stable, but it should be stable as in tests-pass all commits are complete thoughts (unfinished code is not committed), etc. It is API-unstable, and we should probably note or message the difference.

we'll have ~weekly alpha releases off master. Developers can use these but ought to expect things to break once in a while.

All discussions I've seen for this have pointed to one that is normally 6-weekly i.e. after each v8 upgrade.
Stable will of course continue to have weekly releases.

there will be a regular cadence (~weekly?) of stable releases off the stable branch. Developers can (and should) use these and be assured that there will be no breaking changes outside of the occasional breaking security fix. (a stable is guaranteed not to have semver-major bumps... correct?)

Yes, correct. also /occasional/rare/

commits generally land in master, then are cherry picked out to each of the stable/LTS branches.

Correct, unless they aren't able to target master (i.e. fix something that only effected previous versions.)

---

Sounds about right otherwise.

[mikeal]

All discussions I've seen for this have pointed to one that is normally 6-weekly i.e. after each v8 upgrade.

That's strange because the first time I remember hearing that was today :) My assumption has been that these will go out about as often as the stable releases. There's really no reason not to do them regularly as they require the least amount of work compared to all other releases. In fact, there's an argument to be made that we might want to not land any cherrypicked commits from master to stable if they haven't been publicly available for a little while in an alpha release.

[Fishrock123]

Hmmm, ok, I've dug up the references:

It appears that @trevnorris also was operating on this assumption in #1997 (comment) (see paragraph 6.)

And see his plan, in which it was clearly specified: https://gist.github.com/trevnorris/7620a64b086e95271197

Perhaps these were a misunderstanding of @domenic in #1997 (comment) -- where it wasn't specified that it would ONLY be 6 weeks, but rather that there would for sure be releases when v8 was upgraded.

However he now also seems to be understanding this would be 6-week only? http://logs.libuv.org/io.js/2015-07-29#17:42:49.081 (since we'd discussed that releases from the new master branch would probably always be breaking, I think?)

[trevnorris]

master is unstable.

+1 to @Fishrock123 explanation. An individual PR may have a commit that doesn't pass (e.g. updating V8), but with the full PR applied all tests should pass. So it's unstable in terms of API breakage and that it hasn't been widely used by the community so unknown bugs could be lurking.

@Fishrock123 My comment was pointing out the difficulty of only allowing an API incompatible release while at the same time having regular weekly releases. What I meant to say was we need to choose one or the other. Either only have one release every six weeks or have weekly releases and accept that it may break more often than that.

[trevnorris]

In regards to choosing one or the other, I have no strong opinion. As long as we do a release of master that contains the version of V8 that will be in the next major branch several weeks prior to the branch being cut then we offer ample time for projects like nan to make sure everything works before release.

Combine that with the slight overlap of major release support and the community should have ample time to migrate modules. Hopefully diverting mass community breakage, like what we fear will happen with v3.

BTW there are two things that haven't been hammered out yet, and don't know of they need to be at this stage.

1. V8's release won't line up with ours. So on occasion we may need to hold off updating to the latest stable by a week or two so we don't cut a major tag the moment after an update.

2. The initial cut and release of a major (when we begin major support overlap) will be an RC. We'll do weekly RC's for the supported time then simultaneously cut support for the old major and release M.0.0.

[mikeal]

@Fishrock123 ok, but why? What reason do we have not to do regular alpha releases of master? Why is the first release commits in master land in weekly stables they were cherry-picked to rather than regular alphas?

[Fishrock123]

What reason do we have not to do regular alpha releases of master? Why is the first release commits in master land in weekly stables they were cherry-picked to rather than regular alphas?

See the above by @trevnorris:

My comment was pointing out the difficulty of only allowing an API incompatible release while at the same time having regular weekly releases. What I meant to say was we need to choose one or the other. Either only have one release every six weeks or have weekly releases and accept that it may break more often than that.

...

In regards to choosing one or the other, I have no strong opinion.

Same. So long as we can represent it by one number I'm fine.

Something like 5.0.0-upcoming.x or 5.0.0-future.x or 5.0.0-next.x (avoiding "dev", "unstable", "beta", "alpha" and similar wording.)

I think this should be reasonable. It's always going to signify that their could be breakage, since they are using this branch. Obviously we are going to carefully look over semver-major changes that could have large impacts anyways, and be conservative about those.
Plus, there is already little technical reason to differentiate between semver-minor and semver-patch, so I feel those are far less relevant. It will also be easier to message what versions lead up to a stable, and then have the stable releases be predictably versioned.

1. V8's release won't line up with ours. So on occasion we may need to hold off updating to the latest stable by a week or two so we don't cut a major tag the moment after an update.

I've been trying to point this out, yes. There will be times when stable doesn't actually have the latest because the overlap was too short. I think that is fine.

Alternatively we can test against the beta version of v8 and get NAN/etc updated off of that, but seems a little more... risky? Risky isn't the right word though.

[Fishrock123]

Closing, discussion now at #2522 for v4.0.0