Skip to content

The Great JDK Exorcism#310466

Merged
doronbehar merged 3 commits intoNixOS:stagingfrom
Infinidoge:jdk-exorcism
Jul 12, 2024
Merged

The Great JDK Exorcism#310466
doronbehar merged 3 commits intoNixOS:stagingfrom
Infinidoge:jdk-exorcism

Conversation

@Infinidoge
Copy link
Contributor

@Infinidoge Infinidoge commented May 10, 2024

Description of changes

This is a followup to #286267, and that PR must be merged before this one. This PR does not need to target staging, however because it was made to be merged after said PR, it is targeting staging as well. I can rebase and retarget after it makes it to master.

OpenJDK has been an unmaintainable mess in Nixpkgs for quite a while now. In an effort to help rectify this, I've gone ahead and excised several of the JDK's present demons:

  • AdoptOpenJDK, a deprecated project replaced with Adoptium/Temurin and IBM Semeru
  • OpenJDK 12/13/14/15, old, insecure, and in some cases broken and unused versions of OpenJDK
  • The OpenJDK bootstrap chain, which only existed to build those broken versions, and is redundant with their removal

With these packages out of the way, I hope that a bigger refactor of OpenJDK in Nixpkgs is possible.

I hope that this PR and the last will also be a catalyst for a bigger discussion about the state of JDK in Nixpkgs. For being such a big and critical ecosystem, it seems to have a concerning lack of support. I've taken up the mantle currently, but I can only push for so long, and OpenJDK needs to be more maintainable in order to stay up to date in Nixpkgs.

Please let me know if there is anything else that should be removed. Thanks.

On OpenJDK 16

Since openjdk16 is buildable with the binary releases we have in Nixpkgs, I did not see a reason to remove it for the sake of removing it, as it could be kept with little effort. This does keep the dependency on JFX 15, however for the sake of keeping things neat, I've removed openjfx15 from all-packages.nix.

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@ofborg ofborg bot added 8.has: clean-up This PR removes packages or removes other cruft 8.has: package (new) This PR adds a new package labels May 10, 2024
@ofborg ofborg bot requested review from abbradar, edwtjo and taku0 May 10, 2024 02:59
@ofborg ofborg bot added 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 1001-2500 This PR causes many rebuilds on Linux and should target the staging branches. labels May 10, 2024
@lovesegfault
Copy link
Member

cc. @fzakaria

@lovesegfault lovesegfault requested a review from fzakaria May 12, 2024 21:03
@fzakaria
Copy link
Contributor

Cool -- i like the direction as a prior Java Nix user.
Let me know if there's a section you want specific review or feedback.

Given that I found that the JDK is by default not hermetic (it includes default search paths for JNI libraries) -- having so many variants is just a paper cut waiting to happen.

My vote (I know this is very anti-FOSS crowd) would be just to standardize on a single JDK within the nixpkgs (OpenJDK my preference) and offer all the other random variants through some overlay in nix-community.
(headless, AOT, full & minimal could be offered within for the sole OpenJDK)

@niknetniko
Copy link
Contributor

My vote (I know this is very anti-FOSS crowd) would be just to standardize on a single JDK within the nixpkgs (OpenJDK my preference) and offer all the other random variants through some overlay in nix-community. (headless, AOT, full & minimal could be offered within for the sole OpenJDK)

Just a small note (as a user of one of those variants) that at least some variants are probably needed by other packages, for example the JetBrains SDK is necessary to properly run their IDE's, so as long as those are in nixpkgs (as they should be), the JDK will also have to be.

@taku0
Copy link
Contributor

taku0 commented May 20, 2024

I would like to have at least one source based package and one (or more) binary based package, because if something go wrong in one package, we can compare it with the other package.

@Infinidoge
Copy link
Contributor Author

Infinidoge commented May 20, 2024

For my part, I don't think it is necessary or warranted to cull JDK variants in their entirety, as long as they are able to keep being maintained. I don't like removing packages people might depend on unless the cost far outweighs the potential benefit.

AdoptOpenJDK was a specific target because it is a dead and superceded project and stuff it was entangled with was actively becoming unmaintainable. Same with the unneeded bootstrap chain.

The biggest thing JDK in Nixpkgs needs, imo, is more standardization and maintenance scripts to not need to manually look up versions and commit hashes.

@jlesquembre
Copy link
Member

@Infinidoge thanks a lot for this PR, it makes a lot of sense.

For future pull requests, I have one suggestion that could help us move faster: consider keeping each PR focused on a single topic. For example, one PR could target the removal of AdoptOpenJDK, while another could address updating the versions of multiple JDKs.

That said, to avoid any additional work, let's keep this PR as it is.

@Infinidoge
Copy link
Contributor Author

This PR actually already has been split. The linked PR is the one that bumps versions, and is currently open to be merged. This PR just depends on that PR being merged first, which is why it includes its commits.

@doronbehar
Copy link
Contributor

I haven't read the changes in this PR yet. But I wanted to comment on the bootstrap subject.

Guix is known for it's uncompromising bootstrapping conducts, and indeed they bootstrap all the way from java compilers written in c/c++, to modern java compilers.

Here are the relevant sources from their Git repository:

@bendlas
Copy link
Contributor

bendlas commented May 24, 2024

@doronbehar indeed.

I have started porting that: https://github.com/bendlas/nixpkgs/blob/main/pkgs/development/compilers/openjdk/bootstrap-libre/default.nix

@Qyriad Qyriad added the 6.topic: java Including JDK, tooling, other languages, other VMs label May 26, 2024
@Infinidoge
Copy link
Contributor Author

Infinidoge commented Jun 26, 2024

Rebased on top of the latest staging, which significantly trims down this PR by not including the past commits. Since this doesn't cause many rebuilds due to not including the mass rebuild commits from the JDK bump, I'll switch to master once I get the chance (and once OfBorg finishes eval and I properly know how many rebuilds it is)

(Okay now it's rebased on top of latest staging)

@ofborg ofborg bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 501-1000 This PR causes many rebuilds on Linux and should normally target the staging branches. and removed 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-linux: 1001-2500 This PR causes many rebuilds on Linux and should target the staging branches. labels Jun 26, 2024
@Infinidoge
Copy link
Contributor Author

The point of the PR is to remove unmaintainable EOL versions, not EOL versions as a whole (otherwise newer JDKs like 19/20 would also be on the chopping block). JDK 16 has been kept since it is trivial to keep, as opposed to 12, 13, 14, 15 which depend on a long-dead binary distribution and a broken bootstrap chain. Known vulnerabilities takes care of warning people about EOL versions.

Deprecated is a loose term that I am effectively using as shorthand for "kept but shouldn't be used". The general policy I am aiming for is to fully remove only versions that are actively obstructing maintenence to keep.

What makes the decisions about all-packages weird is that openjdk16 has already been removed from all-packages, while openjfx15 hasn't, despite the former being the only use of the latter. Given that, I don't think it should be in all-packages. Which leaves:

  1. Alias (what is present now)
  2. Alias with warning (Adding a lib.warn that says "use javaPackages.openjfx15, this will stop working eventually"
  3. Throw with a removal notice redirecting to above
  4. Outright removal

After this discussion, 1 feels incorrect. 2 feels very overkill considering how many people are likely to be using openjfx15 downstream. This leaves 3 and 4, of which I am in favor of 4, since very very few people are likely to be referring to an outdated library that is already bundled with the built JDK.

If openjdk16 was still in all-packages before the PR, then I would move both to aliases together.

@doronbehar
Copy link
Contributor

I'm too convinced that the situations suggests a removal of openjfx15.

@Infinidoge
Copy link
Contributor Author

As in you suggest removing openjfx15 (and openjdk16) entirely, or do you mean something else?

@doronbehar
Copy link
Contributor

I meant removing their .nix files and put a throwing alias.

Copy link
Contributor

@jerith666 jerith666 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great to me, I appreciate the cleanup efforts, and agree it will lead to easier maintenance in the future.

I'd be okay to see jdk 16 and/or jfx 15 disappear as part of this PR, but also wouldn't hold it up to sort out that relatively minor detail -- it can be done in a separate PR later. :)

Is there a place online where openjdk documents the maintenance status of each version? It'd be nice to link to it somewhere in a comment... I got lost while trying to find such a link.

What JDK packages are considered deprecated and which are considered EOL? Is there a place in which upstream defines this?

https://openjdk.org/projects/jdk-updates/ and https://adoptium.net/support/ seem like the most appropriate places to look regarding FOSS Java updates. But you're right, it's very twisty and hard to get one's head around given all the different providers of OpenJDK builds out there.

Similar to what I said above about 15/16, I wouldn't hold up this PR to get a perfectly refined policy statement on this; that's probably better done in its own PR.

@doronbehar
Copy link
Contributor

I'd be okay to see jdk 16 and/or jfx 15 disappear as part of this PR, but also wouldn't hold it up to sort out that relatively minor detail -- it can be done in a separate PR later. :)

I felt me and @Infinidoge have reached an agreement regarding openjdk16 and openjfx15, so it seems like a matter of day or two until I'll merge this.

openjdk.org/projects/jdk-updates and adoptium.net/support seem like the most appropriate places to look regarding FOSS Java updates.

The 2nd link is excellent due to that roadmap table, and I think it'd be nice to link to it in a comment somewhere in the relevant Nix files, but this too doesn't have to be a blocker. The 1st link OTH seems to mention also EOL releases at the same level as new releases, and doesn't mention which is EOL and which isn't.

@jerith666
Copy link
Contributor

The 1st link OTH seems to mention also EOL releases at the same level as new releases, and doesn't mention which is EOL and which isn't.

If you click through into each release, some have timelines for future releases and some don't.

However, AFAICT, there is no explicit "LTS" designation on Java 21, which most other sources agree is an LTS release.

@wegank wegank added the 2.status: merge conflict This PR has merge conflicts with the target branch label Jul 4, 2024
@thiagokokada
Copy link
Contributor

Any update about this? It would be nice to remove the unmaintained JDK versions.

@Infinidoge
Copy link
Contributor Author

Sorry for the delay, the past 2 weeks have been... hectic for me, between midterms, upcoming appointments, and other life events. I'll go ahead and remove OpenJDK 16 and resolve the merge conflicts now so we can get this merged.

AdoptOpenJDK is a long-deprecated project, having been superceded by
Eclipse Temurin quite a while ago. Additionally, after running the
generate sources command, many of its subpackages fail to evaluate due
to missing binaries for versions the package expects. Because everything
provided by AdoptOpenJDK is either long-deprecated or also provided by
Temurin, its removal should not cause many problems.

By the same token, OpenJDK 12, 13, 14, 15, and 16 have also all been long
deemed EOL, and 13/14 are both actively broken and fail to build. These
packages, and their associated (and unnecessary) bootstrap chain are a
major factor in the tech debt of OpenJDK as an ecosystem in Nixpkgs.

OpenJDK 16 was the only user of OpenJFX 15, so it has also been removed.

By removing these packages, OpenJDK should hopefully be more
maintainable into the future.
@ofborg ofborg bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Jul 11, 2024
@thiagokokada
Copy link
Contributor

Eval error:


         at /ofborg/checkout/3/repo/38dca4e3aa6bca43ea96d2fcc04e8229/mr-est/ofborg-evaluator-3/pkgs/games/minecraft-servers/default.nix:8:20:

            7|
            8|   getJavaVersion = v: (builtins.getAttr "openjdk${toString v}" javaPackages.compiler).headless;
             |                    ^
            9|

       error: attribute 'openjdk16' missing

@Infinidoge
Copy link
Contributor Author

Infinidoge commented Jul 11, 2024

Amazing. I love how it hides dependencies.

I'll just make it substitute openjdk16 for openjdk17.

@doronbehar
Copy link
Contributor

There is currently a CI error on staging unrelated to this PR, sorry about that. Once #326478 is merged I'll look into this once more. Changes look good overall.

@doronbehar
Copy link
Contributor

@ofborg eval

@doronbehar
Copy link
Contributor

Another eval error unrelated to this PR, sorry again, waiting for #326491 which will hopefully fix the eval error.

@doronbehar
Copy link
Contributor

@ofborg eval

@ofborg ofborg bot added 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-darwin: 1 This PR causes 1 package to rebuild on Darwin. and removed 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. labels Jul 12, 2024
@doronbehar doronbehar merged commit d88f297 into NixOS:staging Jul 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.topic: java Including JDK, tooling, other languages, other VMs 8.has: clean-up This PR removes packages or removes other cruft 8.has: package (new) This PR adds a new package 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-darwin: 1 This PR causes 1 package to rebuild on Darwin. 10.rebuild-linux: 501-1000 This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches.

Projects

None yet

Development

Successfully merging this pull request may close these issues.