graalvm-oracle: init at 22

[farnoy]

Supersedes #259639, resolves #259631

Description of changes

This adds oracle-graalvm and make buildGraalvm more generic to be used on other places than community-edition.

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
  • sandbox = relaxed
  • sandbox = true
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

Pinging @thiagokokada (reviewer from the previous PR)

---

Add a 👍 reaction to pull requests you find important.

[farnoy]

This used to be called GraalVM EE, now just "Oracle GraalVM". Maybe oracle-graalvm would be a better name here?

[rafaelrc7]

Do you plan on adding the other supported versions by oracle's graalvm? Such as for JDK 17 and 21.

[farnoy]

@rafaelrc7 I don't have any use for them but I'll take a look

[farnoy]

Rebased on latest nixpkgs to resolve conflicts

[rafaelrc7]

@rafaelrc7 I don't have any use for them but I'll take a look

Thanks! I am currently working on packaging pkl in #286658 and the native build depends on graalvm 17, so it would be a lot of help.

[thiagokokada]

I don't like that we have 2 different graalvmPackages.

I think since we are introducing Oracle GraamVM again, it is a good time to move everything to graalvmPackages and make graalvmCEPackages an alias to graalvmPackages, and deprecate and eventually remove graalvmCEPackages.

[farnoy]

I can do that, but can you be more specific? Do we want graalvmPackages to temporarily be the CE packages and also graalvmPackages.oracle + graalvmPackages.ce as the long-term, stable names?

Do we also want this nesting for JDK packages or is pkgs.oracle-graalvm fine as it is in the current version of the PR?

[thiagokokada]

I can do that, but can you be more specific? Do we want graalvmPackages to temporarily be the CE packages and also graalvmPackages.oracle + graalvmPackages.ce as the long-term, stable names?

Yes. But probably the names will be graalvmPackages.graalvm-oracle and graalvmPackages.graalvm-ce, because there are other packages inside graalvmCEPackages too.

[farnoy]

I've updated the PR, please let me know what you think

[farnoy]

Not sure if this URL is deterministic, there is no /archive/ version for these

[farnoy]

I think these hashes are correct, I calculated them with nix-prefetch-url but I need to double check.

[farnoy]

Thanks for the feedback @chayleaf , updated

[chayleaf]

Suggested change

	graalvmCEPackages = graalvmPackages;
	graalvm-ce = graalvmPackages.graalvm-ce;
	graalvmCEPackages = graalvmPackages; # 2024-08-10

aliases.nix is for deprecated aliases, so you have to put the date in there as a rough estimate of whether it's okay to replace the alias with a throw (see maintainers/scripts/remove-old-aliases.py)

graalvm-ce is probably fine to not deprecate? in that case, just put it back to all-packages.nix

[farnoy]

graalvm-ce now lives in graalvmPackages.graalvm-ce so I'm deprecating this old name #321026 (comment)

[thiagokokada]

Those URLe are unacceptable, especially in this case since Hydra will not cache the result (because this is unfree).

We need either to have a stable URL from upstream or have the files archived somewhere (e.g.: archive.org).

[chayleaf]

there are some precedents, but they typically result in the packages being marked as broken (see fmod)

[farnoy]

I found more stable links (with full version strings) here.

I'm not sure if this is sufficient from what you're saying. This package will always be unfree, regardless where we download it from? And I'm not sure it's legal to replicate it to archive.org.

[chayleaf]

the ones with full version strings should be sufficient (there are some for 17 as well)

[thiagokokada]

You are adding 2 separate versions but the commit only says graalvm-oracle: init at 22.

[farnoy]

I've split it up into two commits. I'll do some updates and introduce version 23 in a follow up PR

[thiagokokada]

nixpkgs-review result

Generated using nixpkgs-review.

Command: nixpkgs-review pr 321026

---

x86_64-linux

❌ 8 packages failed to build:

• babashka
• babashka-unwrapped
• bbin
• neil
• tests.writers.bin.babashka
• tests.writers.simple.babashka
• tests.writers.wrapping.babashka
• tests.writers.wrapping.babashka-bin

✅ 20 packages built:

• certificate-ripper
• clj-kondo
• cljfmt
• clojure-lsp
• cq
• dapl-native
• dbqn-native
• graalvmPackages.graaljs
• graalvmPackages.graalnodejs
• graalvmPackages.graalpy
• graalvmPackages.graalvm-ce
• graalvmPackages.graalvm-ce-musl
• graalvmPackages.graalvm-oracle (graalvmPackages.graalvm-oracle_22)
• graalvmPackages.graalvm-oracle_17
• graalvmPackages.truffleruby
• jet
• scala-update
• vscode-extensions.betterthantomorrow.calva
• yamlscript
• zprint

[farnoy]

@thiagokokada

So babashka fails on this test: https://github.com/NixOS/nixpkgs/blob/394571358ce82dff7411395829aa6a3aad45b907/pkgs/development/interpreters/babashka/default.nix#L38C1-L39C1

$out/bin/bb '(prn "bépo àê")' | fgrep 'bépo àê'

It outputs "b?po ??" on my end, is my system's locale/encoding leaking somewhere?

I tried it on my other NixOS machine and it works:

~$ nix build "github:farnoy/nixpkgs?branch=oracle-graalvm#babashka"
~ 35s $ nix eval "github:farnoy/nixpkgs?branch=oracle-graalvm#babashka"
«derivation /nix/store/18ikvgm7qy41vkwhjyc8zmvyhic2i28a-babashka-1.12.194.drv»
~$ nix log /nix/store/18ikvgm7qy41vkwhjyc8zmvyhic2i28a-babashka-1.12.194.drv | grep '"b'
warning: The interpretation of store paths arguments ending in `.drv` recently changed. If this command is now failing try again with '/nix/store/18ikvgm7qy41vkwhjyc8zmvyhic2i28a-babashka-1.12.194.drv^*'
"bépo àê"

I don't know how to debug this deeper than this, but I think something from the environment is leaking

[thiagokokada]

This file still seem to be missing the latest changes from master.

[farnoy]

This is the parent of my commits 3b74ae1

I don't think I'm missing anything

[thiagokokada]

Just compare this file with the one from master: https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/compilers/graalvm/community-edition/buildGraalvm.nix.

It is pretty clear that something went wrong. Probably because you moved the file before the changes.

[thiagokokada]

Oh I know the reason, your PR introduces another file called buildGraalvm in another path and forgot to delete the previous one (the one inside community-edition). So it didn't get any conflicts (because for git there is no conflicts, just a new file) and now we are building from the wrong file.

[thiagokokada]

Or in other words, this should appear in the diff as a move, not as a new file.

[thiagokokada]

nixpkgs-review result

Generated using nixpkgs-review.

Command: nixpkgs-review pr 321026

---

x86_64-darwin

⏩ 3 packages marked as broken and skipped:

• dapl-native
• dbqn-native
• obb

❌ 17 packages failed to build:

• babashka
• babashka-unwrapped
• bbin
• certificate-ripper
• clj-kondo
• cljfmt
• clojure-lsp
• cq
• graalvmPackages.graalvm-ce
• graalvmPackages.graalvm-oracle
• graalvmPackages.graalvm-oracle_17
• jet
• neil
• scala-update
• vscode-extensions.betterthantomorrow.calva
• yamlscript
• zprint

✅ 4 packages built:

• graalvmPackages.graaljs
• graalvmPackages.graalnodejs
• graalvmPackages.graalpy
• graalvmPackages.truffleruby

[farnoy]

@ofborg build jet babashka clj-kondo

I don't have a darwin machine to test locally

[farnoy]

There's something non-deterministic about it. I'm able to build babashka on my branch, on the machine that got the encoding error before.

$ nix build "github:farnoy/nixpkgs?branch=oracle-graalvm#babashka" --no-substitute
1m 22.9s $ nix eval "github:farnoy/nixpkgs?branch=oracle-graalvm#babashka"«derivation /nix/store/18ikvgm7qy41vkwhjyc8zmvyhic2i28a-babashka-1.12.194.drv»
$ nix log /nix/store/18ikvgm7qy41vkwhjyc8zmvyhic2i28a-babashka-1.12.194.drv
warning: The interpretation of store paths arguments ending in `.drv` recently changed. If this command is now failing try again with '/nix/store/18ikvgm7qy41vkwhjyc8zmvyhic2i28a-babashka-1.12.194.drv^*'
Running phase: patchPhase
@nix { "action": "setPhase", "phase": "patchPhase" }
Running phase: updateAutotoolsGnuConfigScriptsPhase
@nix { "action": "setPhase", "phase": "updateAutotoolsGnuConfigScriptsPhase" }
Running phase: configurePhase
@nix { "action": "setPhase", "phase": "configurePhase" }
no configure script, doing nothing
Running phase: installPhase
@nix { "action": "setPhase", "phase": "installPhase" }
Running phase: fixupPhase
@nix { "action": "setPhase", "phase": "fixupPhase" }
shrinking RPATHs of ELF executables and libraries in /nix/store/rc83vlda57811c6yld7hb6f70b6z9d9s-babashka-1.12.194
checking for references to /build/ in /nix/store/rc83vlda57811c6yld7hb6f70b6z9d9s-babashka-1.12.194...
patching script interpreter paths in /nix/store/rc83vlda57811c6yld7hb6f70b6z9d9s-babashka-1.12.194
Running phase: installCheckPhase
@nix { "action": "setPhase", "phase": "installCheckPhase" }
babashka v1.12.194
3
[1 2]
"bépo àê"
"ä\n"
#object["[Ljava.lang.Object;" 0x2c040989 "[Ljava.lang.Object;@2c040989"]
Clojure CLI version (deps.clj) 1.12.0.1479

[thiagokokada]

There's something non-deterministic about it. I'm able to build babashka on my branch, on the machine that got the encoding error before.

This non-deterministic behavior was fixed a long time ago in master in buildGraalvm but your PR doesn't reflect those changes. Please update it to use the correct version of the file.

[farnoy]

Good catch, thanks. That file is gone now and there's only the one buildGraalvm.nix that's fully up to date with master

[thiagokokada]

nixpkgs-review result

Generated using nixpkgs-review.

Command: nixpkgs-review pr 321026

---

x86_64-linux

✅ 8 packages built:

• graalvmPackages.graaljs
• graalvmPackages.graalnodejs
• graalvmPackages.graalpy
• graalvmPackages.graalvm-ce
• graalvmPackages.graalvm-ce-musl
• graalvmPackages.graalvm-oracle
• graalvmPackages.graalvm-oracle_17
• graalvmPackages.truffleruby

[thiagokokada]

Code-wise, LGTM.

Will run nixpkgs-review for aarch64-darwin and x86_64-darwin, and also wait until the eval checks pass to merge.

[thiagokokada]

nixpkgs-review result

Generated using nixpkgs-review.

Command: nixpkgs-review pr 321026

---

x86_64-darwin

✅ 7 packages built:

• graalvmPackages.graaljs
• graalvmPackages.graalnodejs
• graalvmPackages.graalpy
• graalvmPackages.graalvm-ce
• graalvmPackages.graalvm-oracle
• graalvmPackages.graalvm-oracle_17
• graalvmPackages.truffleruby

[farnoy]

What a journey! Thanks. I don't know what's going with ofborg, I haven't seen it be green anytime recently

[thiagokokada]

nixpkgs-review result

Generated using nixpkgs-review.

Command: nixpkgs-review pr 321026

---

aarch64-darwin

✅ 7 packages built:

• graalvmPackages.graaljs
• graalvmPackages.graalnodejs
• graalvmPackages.graalpy
• graalvmPackages.graalvm-ce
• graalvmPackages.graalvm-oracle
• graalvmPackages.graalvm-oracle_17
• graalvmPackages.truffleruby

[thiagokokada]

What a journey! Thanks. I don't know what's going with ofborg, I haven't seen it be green anytime recently

It is slower, but still running. It should take 1 or 2 days to eval your PR, but it should happen eventually.

[farnoy]

@thiagokokada it's green, can you merge?