2+2+2 Java Support Plan

[MarkEWaite]

2+2+2 Java Support Plan

Draft documents and other supporting information

• Java 11, 17, and 21 in Jenkins
• Visualizing Java versions for Jenkins
• Jenkins developer mailing list discussion
• Jenkins user mailing list discussion

Governance board video discussion of the proposal

Would like review from:

• @Wadeck - security officer
• @dduportal - infrastructure officer
• @kevin27 - documentation officer
• @timja - release officer
• @basil - core maintainer
• @uhafner - board member
• @NotMyFault - board member
• @jtnord - core maintainer
• @daniel-beck - security team
• @jleon33 - CloudBees product manager

None of the reviews are required, but I would be very happy to have inputs and suggestions.

[samrocketman]

Any plans for Groovy? By default it outputs Java 1.5 bytecode or if you customize the output at most it will output Java 1.8 bytecode when compiling Groovy classes.

[MarkEWaite]

Any plans for Groovy? By default it outputs Java 1.5 bytecode or if you customize the output at most it will output Java 1.8 bytecode when compiling Groovy classes.

Thanks for asking. No plans for Groovy that are part of this JEP. Groovy upgrade is a major project that will need to be considered separately.

[MarkEWaite]

@timja thanks for reviewing it. Do you feel that we're at a point where this can is ready for approval as a draft JEP? I'm a JEP editor, so I'm happy to perform the necessary steps. The approval process says:

A JEP editor will check the submission for conformance with JEP structure and formatting guidelines. Editors may make minor changes to make the submission meet the requirements for approval as a Draft JEP. If a JEP requires major changes, editors will add specific feedback and send the submission back to the sponsor for revision.
...
Once JEP meets requirements for structure and formatting, the editors will approve the submission as a draft JEP by following the steps outlined in the editors' "Approve as Draft" section. When they are done, the Draft JEP will have an official JEP number and the submission PR will have been merged to a matching folder (for example, jep/1) in the master branch.

If you (as a JEP editor) feel that it is ready for approval as draft, then I'll follow the steps in the "Approve as draft" section so that the proposal is assigned a JEP number.

I'm also fine if we need to wait for review by others before it is assigned a JEP number and listed as a draft JEP.

[timja]

Yes fine to assign a number etc go for it

[jtnord]

this seems rather agressive to the point of anoying.
if a user follows the minimum supported version they would have 6 months only without any "nag screen". if we are supporting it for 2 years as a minimum version then it is my belief that they should be able to run the minimum version without nag for much more than 6 months.

[MarkEWaite]

if we are supporting it for 2 years as a minimum version then it is my belief that they should be able to run the minimum version without nag for much more than 6 months.

I'm open to alternative dates. Whatever warning time period that we choose is striking a compromise between alerting too early and not alerting early enough. Some alternate scenarios for dates:

Current warning - 18 months and 9 months

• warn 18 months before end of Java version support by Jenkins
• danger 9 months before end of Java version support by Jenkins

This accepts the warning is well before the end of Java version support. The administrative monitor is easy to clear. Enterprise users have told me that they need 9 months of transition time from one Java version to the next Java version. This is trying to give them that extra warning time, accepting that with Java 17 and Java 21, the administrative monitor will appear in the same LTS release that make those Java versions the required version.

I hope that that 18 month warning will persuade more Jenkins users to switch from the minimum required Java version to the next Java version. When Java 17 becomes the minimum version, they already look carefully at Java 21 and begin their transition plans.

One year warning - 12 months and 6 months

• warn 12 months before end of Java version support by Jenkins
• danger 6 months before end of Java version support by Jenkins

This prefers to delay the administrative monitor so that users have a longer period on the minimum Java version before they are reminded to upgrade to the next Java version.

15 month warning - 15 months and 9 months

• warn 15 months before end of Java version support by Jenkins
• danger 9 months before end of Java version support by Jenkins

This gives 3 more months before the user of a minimum Java version is warned and still provides the danger message 9 months before the end of support for that Java version.

@jtnord is one of those scenarios better in your view than the others?

[jtnord]

A monitor at 12 months should IMO given users enough time to start their migration (if they need a lot of planning and agent migration - this is the warning for them). I feel any earlier will be a case of - "we have plenty of time to not action this for now..", and fall into the needing the second alert. If a migration would take 9 months for a user, then they have been given a full quarter notice to plan this.

Thus I would think 12 and 6 months would be better. (I would not be against a 12 and 9 month).

I hope that that 18 month warning will persuade more Jenkins users to switch from the minimum required Java version to the next Java version. When Java 17 becomes the minimum version, they already look carefully at Java 21 and begin their transition plans.

Also considering we have not swapped anyone using the default (javaless) container until 18 months as currently stated - I'm not sure we would have critical mass to know that this would be a good thing or not at 18 months?

[timja]

I agree with James, 12 and 6 months seems better.

[daniel-beck]

12 and 3? That's still one full quarter and LTS cycle, something we expect admins to (mostly) track with their updates.

We should also remember that this only matters for people generally closely tracking latest releases (~security fixes) anyway; if they go half a year without applying any updates, their inability to do so at the cutoff doesn't really matter.

[MarkEWaite]

18/9, or even 12/6 errs too far on the side of those for whom this is extreme effort (and who dismissed the first message more appropriate for their environment!), while showing everyone else completely irrelevant notices they won't care about for many months to come.

One of the intents in showing the warning earlier (18 months instead of 12 for first warning) is to persuade administrators to make the change sooner. The earlier we show the message, the sooner administrators will start the transition from the previous JDK to the new JDK. However, I understand that many administrators may ignore a warning at 18 months. I'm open to 12/6 or even 12/3.

[daniel-beck]

The earlier we show the message, the sooner administrators will start the transition from the previous JDK to the new JDK.

I'm not sure that's actually the case. "Do work now to get a benefit in 18 months" in not a compelling message. It's a useful heads up for folks who have more complex upgrade processes (and even then 18 months seems excessive, acknowledging this is a situation I'm not familiar with), but for anyone with more straightforward upgrade paths, I expect the earlier this is shown, the more likely this is going to just be ignored.

[MarkEWaite]

The Java adoption curves for Jenkins installations shows that the transition from Java 8 to Java 11 accelerated significantly after the admin monitor became visible in Jenkins core. It was visible in a weekly release a little over 12 months prior to the weekly release that required Java 11.

The Java 11 adoption curve prior to the admin monitor was growing nicely, but the slope of the curve increased after the admin monitor became visible. The Java 8 adoption curve was already flattening and even decreasing before the admin monitor, but the rate of decrease increased significantly after the admin monitor became visible.

Since the Java 8 to Java 11 monitor appeared about 12 months before Java 11 became the minimum Java version, it seems reasonable that we use 12 months for the initial warning for future Java transitions.

We raised a second admin monitor with the Java 11 transition though the adoption curve does not show any obvious change of slope from the second admin monitor. Based on that, I'm fine with either 12/6 or 12/3.

@jtnord are you OK with 12/3 rather than 12/6?
@timja are you OK with 12/3 rather than 12/6?

I'd like to get that change into Jenkins 2.430 on Tuesday so that we can consider it for backport into 2.426.1.

[MarkEWaite]

Pull request submitted to Jenkins core that implements 12/3:

• [JENKINS-72252] Warn 12 months and 3 months before end of Java support jenkins#8661

[MarkEWaite]

I've updated the JEP in bc6ad48 to use 12 and 3.

[jtnord]

see above - seems a bit aggressive? 9 months is 3 approx. 3 Jenkins LTS version upgrades.
if we are supporting this version (which we are) - then this seems to be overly aggressive for a "supported" JDK

[jtnord]

the phases here could be so much easier to visualize with something like mermaid and a gannt chart.

[MarkEWaite]

Agreed. That's why I used the spreadsheet as an impoverished diagramming tool. I hope to create a diagram that expresses the details more clearly.

[MarkEWaite]

Two diagrams have been added to show the phases.

[basil]

I have little interest in the discussion about when to display the administrative monitors. The arguments that have been presented seem fairly subjective to me. Are they examples of the law of triviality?

The more pressing issue from my perspective is rather that this JEP does not have a prototype implementation. Per JEP-1:

Standards Track JEPs consist of two parts, a design document and a prototype implementation. The prototype implementation should be co-developed with the JEP, as ideas that sound good in principle sometimes turn out to be impractical when subjected to the test of implementation.

This is a critical part of the JEP process from my perspective. The fact that it is part of the JEP process reflects the deep wisdom of its designers. How, then, could this PR contain the text

No prototype implemented

The work to drop support for Java 8 cannot really be considered an example, as some parts apply to Java 9+ and others do not. The work to add support for Java 21 and to recommend Java 17 can be considered as examples, but this JEP is quite vague regarding what general conclusions should be drawn from these examples.

Since the purpose of this JEP is to institute a new process, I believe that the prototype implementation should be documentation of this new process in the form of checklists, gleaning the relevant insights from the past (where applicable) and proposing new procedures for the future (when past precedent is not applicable).

If this is asking too much from the JEP author, then I would be happy to prepare a competing JEP that addresses this criticism. However, my hope is that we could collaborate together rather than compete in two JEPs, so I have prepared the following commit as an example of what I am requesting:

basil/release@f49ad9c

In other words, I am asking that this commit be fleshed out and linked to as the prototype implementation of the process in this JEP.

It is fairly complete, but there are a few remaining TODOs. Some of these represent the interesting scenarios where we have little to no past precedent. For example, how should we drop support for a given Java version in the Java 9+ world? We have only ever done this in the past for Java 8, and I do not think many of the constraints there are applicable to Java 9+. So this is an area where we at least need to revisit past precedent, if not define a new procedure. It occurs to me that several paths are possible, none of which are discussed in this JEP, nor is there any existing precedent that can be exactly copied.

I am hitting the "Request Changes" button on this PR specifically for this last issue: the question of how to drop support for Java versions in the future with regard to the build toolchain is an important design conversation that I believe should happen in the context of this JEP rather than postponed until later. Experience shows that it is better to get disagreement and debate out of the way upfront rather than to have it once the implementation of a project is underway.

I am also reiterating that I would rather collaborate on this JEP than to write a competing JEP that meets my standards, so please let me know how I can help.

[MarkEWaite]

The more pressing issue from my perspective is rather that this JEP does not have a prototype implementation. Per JEP-1:

Standards Track JEPs consist of two parts, a design document and a prototype implementation. The prototype implementation should be co-developed with the JEP, as ideas that sound good in principle sometimes turn out to be impractical when subjected to the test of implementation.

This is a critical part of the JEP process from my perspective. The fact that it is part of the JEP process reflects the deep wisdom of its designers. How, then, could this PR contain the text

I agree that a prototype is a critical part of the JEP process for standards track JEPs and I very much like your recommendation that we should improve this JEP with checklists for key events like adding a Java version and removing a Java version. I look forward to working with you on the checklists as a way to make this JEP more useful and more effective.

For the sake of precision, I think that a prototype is not strictly required because this is a process JEP as described in the JEP types where it says:

A Process JEP describes a process surrounding Jenkins, or proposes a change to (or an event in) a process. Process JEPs are like Standards Track JEPs but apply to areas other than the Jenkins codebase itself. They may propose an implementation, but not for what would be generally considered the Jenkins codebase; they often require community consensus; unlike Informational JEPs, they are more than recommendations, and users are typically not free to ignore them. Examples include procedures, guidelines, changes to the decision-making process, and changes to the tools or environment used in Jenkins development. Any meta-JEP (JEP focusing on the improvement of the Jenkins Enhancement Proposal process) is also considered a Process JEP.

It may be that I incorrectly assigned this as a process JEP instead of a standards track JEP, but the things that I was trying to describe seemed to better fit "procedures, guidelines, changes to the decision-making process, and changes to the tools or environment used in Jenkins development" than "a new feature or implementation for Jenkins".

Since the purpose of this JEP is to institute a new process, I believe that the prototype implementation should be documentation of this new process in the form of checklists, gleaning the relevant insights from the past (where applicable) and proposing new procedures for the future (when past precedent is not applicable).

I like that idea very, very much. I'm happy to work with you to describe the stages and am happy to link to https://github.com/basil/release/tree/java-lifecycle as proposed checklists for the stages of the Java lifecycle.

• Add a new Java version
• Recommend a new Java version
• Require a new Java version

I see a strong connection between "Require a new Java version" and "Drop support of an old Java version". I assume that the description of "Drop support of an older Java version" will be inside the "Require a new Java version" checklist, since requiring a new Java version infers the end of support for the older Java version. Let me know if I'm wrong on that.

[basil]

I agree that a prototype is a critical part of the JEP process for standards track JEPs and I very much like your recommendation that we should improve this JEP with checklists for key events like adding a Java version and removing a Java version. I look forward to working with you on the checklists as a way to make this JEP more useful and more effective.

Thanks Mark, I am glad we agree on the basic premise and I am happy to help make this a reality.

For the sake of precision, I think that a prototype is not strictly required because this is a process JEP as described in the JEP types […] It may be that I incorrectly assigned this as a process JEP instead of a standards track JEP, but the things that I was trying to describe seemed to better fit "procedures, guidelines, changes to the decision-making process, and changes to the tools or environment used in Jenkins development" than "a new feature or implementation for Jenkins".

I am not sure, and I am not particularly interested in debating the letter of the law, either. From my perspective, the spirit of the law is that architects are obligated to reify their abstract proposals with concrete prototypes in order for the abstract proposals to have merit. This has not yet been done for this JEP, and the result is that some critical design areas remain ambiguous, so my blocking feedback remains.

I like that idea very, very much. I'm happy to work with you to describe the stages and am happy to link to https://github.com/basil/release/tree/java-lifecycle as proposed checklists for the stages of the Java lifecycle.

Well, I was not volunteering to drive the prototype implementation myself, at least not within the context of the current PR, of which I am not the author. I was merely offering the above as a starting point to help the current PR's author (i.e. you) reify their proposal and thereby bring it up to the standard by which I would approve a JEP. If you do not have the time or interest to drive this, then that is quite alright, but in that case I will file a competing JEP that does meet my standards, including having a complete prototype implementation in the form of the checklists I described, with the work being driven by me. Sorry, but I strongly believe there needs to be a connection between architecture and implementation.

I see a strong connection between "Require a new Java version" and "Drop support of an old Java version".

Yes, I used these terms interchangeably in my post above, preferring one term over the other simply to emphasize the point that the next time we do this will be quite different from the last time we did this.

[MarkEWaite]

I was merely offering the above as a starting point to help the current PR's author (i.e. you) reify their proposal and thereby bring it up to the standard by which I would approve a JEP. If you do not have the time or interest to drive this, then that is quite alright, but in that case I will file a competing JEP that does meet my standards, including having a complete prototype implementation in the form of the checklists I described, with the work being driven by me.

I have the interest and I hope to have the time to drive it. If I find that I don't have the time to drive it, I will certainly let you know so that I can close this pull request and let you submit a pull request with the desired checklists.

I've already found great value in my reviews of the tickets and pull requests that are examples of "Add a new Java version".

I realize that your blocking comment is on the "Drop support for a Java version", but I want to capture the easier checklist first. I think that the detailed checklist for "Add a Java version" will be good seed material for the "Drop support for a Java version" checklist.

[basil]

I am hitting the "Request Changes" button on this PR specifically for this last issue: the question of how to drop support for Java versions in the future with regard to the build toolchain is an important design conversation that I believe should happen in the context of this JEP rather than postponed until later. Experience shows that it is better to get disagreement and debate out of the way upfront rather than to have it once the implementation of a project is underway.

I thought about the "require a new Java version" problem some more today and came to some concrete points:

• Unlike the changes to require Java 11, the POM changes to require Java 17 are much less invasive: at the present time, everything is defined in terms of the maven.compiler.release and maven.compiler.testRelease properties (currently set to 11). We could even define maven.compiler.testRelease in terms of maven.compiler.release if desired to bring the number of property knobs down from 2 to 1.
• With the changes to require Java 11, I attempted to write logic in maven-hpi-plugin to dynamically set Maven properties based on the Jenkins core version, but it was impractical. While it worked for command-line builds, it generally did not work for IDEs and thus caused more confusion. While Maven does allow you to set Maven properties from within a Maven plugin mojo, IDEs generally don't run these mojos and will therefore use the defaults instead.
• The current interface between parent POMs and consumers is for the parent POM to declare the maven.compiler.release and maven.compiler.testRelease properties and for the consumer not to declare anything. One option would be to follow a similar process to the one we followed for requiring Java 11, where we release new parent POMs that require Java 17 and accept that plugins with an older baseline (which, on day 1, will be all of them) won't be able to get parent POM updates until they start requiring a newer baseline, likely 3-4 months down the road. I am not sure if that is a precedent that we want to set for the long term, since breaking parent POM updates is a pretty big disadvantage. We did manage to tolerate this strategy last time around, in part because of my Maven HPI plugin hack to dynamically set the property (which didn't work in IDEs).
• Another option is go back to the older system of plugin consumers declaring both a Jenkins core version and a Java version. We might want to add Maven HPI plugin logic to validate the consumers aren't trying to do something unsupported, like setting the Java version to anything other than the one that matches the bytecode level of their core baseline. In that case we'd need to do a sweep to re-add that property to popular plugins. Advantages: avoids the flag day mentioned above. Disadvantages: requires a sweep to implement, harder for maintainers to keep these properties in sync and less obvious what the values should be. Though maybe some of these disadvantages could be mitigated if Maven HPI plugin enforced a specific value and failed the build otherwise, at least making it obvious what these values should be?
• There could be other options that I haven't considered. For example, maybe I was overly harsh in my assessment of my previous hack to have Maven HPI plugin dynamically set the property based on the Jenkins core version, even though IDEs won't pick it up. Maybe we adopt one of the options recommended previously but reinstate this hack as a fallback? This is a variation of the previous bullet point, but instead of being a hard error, the Maven HPI plugin would make a best effort attempt to fix up the Java version property, which would likely succeed in CLI builds but fail for IDE builds.

Anyway, this is an example of the type of thing that I consider important to cover within this JEP in order to ensure the completeness of the design. If we don't figure out the plan for things like this upfront, then we'll just have to solve these problems as we encounter them, and by then it may be too late to implement some of these solutions, because it takes a while to propagate HPI plugin and POM changes throughout the ecosystem.

[user8472]

Am I reading this thread correctly, I find no mention that you are removing Support for the Solaris platform,
Java 11 is the last currently supported version java that will be supported on Solaris.

[MarkEWaite]

Am I reading this thread correctly, I find no mention that you are removing Support for the Solaris platform, Java 11 is the last currently supported version java that will be supported on Solaris.

Oct 2024 is the planned end of life for all Java 11 support in Jenkins. Since Java 17 and Java 21 are not supported on Solaris, Jenkins will not be supported on Solaris after the end of Java 11 support in Jenkins.

[basil]

As far as I know Peter Tribble is still maintaining an illumos/Solaris OpenJDK port, and Peter's port of OpenJDK 21 is available in his "Tribblix" distribution. I cannot comment on builds/packaging of this port to other distributions, as I am no longer familiar with work in this area.

[MarkEWaite]

As far as I know Peter Tribble is still maintaining an illumos/Solaris OpenJDK port, and Peter's port of OpenJDK 21 is available in his "Tribblix" distribution. I cannot comment on builds/packaging of this port to other distributions, as I am no longer familiar with work in this area.

That's great news!

[basil]

Per the maintainer documentation I am applying the stalled label to this pull request, as it has remained inactive for a month.