Closed
Conversation
Member
Author
|
This PR is a draft because I'm running into some difficulties to package it. I solved lots of errors, but the program still not builds. Currently it errors with: The code seems to be downloading this dependency. The problem seems to be that this task, for some reason, is not using the local repo. |
Closed
ofborg
bot
added
8.has: package (new)
Member
|
According to the Pkl installation instructions, the java executable has a noticeable startup delay and runs complex code slower than the native executables. It would be great if this derivation could produce the native executables for supported platforms instead. |
Member
Author
Indeed it would... However, the native executable depends on the java build process, so I am trying to fix it first. And the native build also depends on GraalVM for JDK 11, that seems to be a problem, as it was removed from nixpkgs some months ago because it got unsupported. So I'm not sure how to deal with that. When I tried building with the graalvm package currently available (for JDK 21), it did not work, maybe it is fixable, but, again, I'm having trouble with the java build, as described above. |
Member
|
I'd also love to see support for |
Member
Author
|
I also asked about the gradle issues in the pkl repo apple/pkl#90 . I'm waiting for suggestions either here or there. Also trying some stuff by myself, but I did not have much luck. |
Member
|
Last week the pkl repo issue was updated with more information |
Member
|
Pkl 0.26.0 has been released. |
Contributor
|
#272380 got merged, so you can give this a second shot now! See https://github.com/NixOS/nixpkgs/blob/master/doc/languages-frameworks/gradle.section.md |
Member
Author
|
Thanks for the work, @chayleaf ! It really helped with gradle packages and eased a lot of the verbosity. However, this package still seems to face the same problems... I've done as you tell in the manual page you wrote, and rewrote the And this dependency ( As a test, I tried removing this dependency from the project, by adding: to the derivation. However, this resulted in another error: This one, however, seems to have been add to the dependency file: So I am not sure what the issue is here... I notice the second item has the pom hash, but not the jar hash, is this related? |
Contributor
|
This is using spotless, a plugin that's using dynamic dependencies for code formatting/etc, which makes no sense for a normal build. You should disable it using |
Contributor
|
Fixed the build: diff --git a/pkgs/by-name/pk/pkl/package.nix b/pkgs/by-name/pk/pkl/package.nix
index 1142728572e0..3c8a8a1e4e32 100644
--- a/pkgs/by-name/pk/pkl/package.nix
+++ b/pkgs/by-name/pk/pkl/package.nix
@@ -14,13 +14,24 @@ stdenv.mkDerivation rec {
owner = "apple";
repo = "pkl";
rev = version;
- sha256 = "sha256-9YXccQloPxGekkR78tvkWKCmB9wXzL7iFkRL4yLQVZQ=";
+ sha256 = "sha256-Q7B6DRKmgysba+VhvKiTE98UA52i6UUfsvk3Tl/2Rqg=";
+ # the build needs the commit id, replace it in postFetch and remove .git manually
+ leaveDotGit = true;
+ postFetch = ''
+ cd "$out"
+ export commit_id=$(git rev-parse --short HEAD)
+ cp ${./set_commit_id.patch} set_commit_id.patch
+ chmod +w set_commit_id.patch
+ substituteAllInPlace set_commit_id.patch
+ git apply set_commit_id.patch
+ rm set_commit_id.patch
+ find "$out" -name .git -print0 | xargs -0 rm -rf
+ '';
};
nativeBuildInputs = [
git
gradle
- jdk17
];
buildInputs = [
@@ -34,9 +45,13 @@ stdenv.mkDerivation rec {
__darwinAllowLocalNetworking = true;
- gradleFlags = [ "-Dfile.encoding=utf-8" ];
+ gradleFlags = [
+ "-x" "spotlessCheck"
+ "-DreleaseBuild=true"
+ "-Dorg.gradle.java.home=${jdk17}"
+ ];
- gradleBuildTask = "build";
+ JAVA_TOOL_OPTIONS = "-Dfile.encoding=utf-8";
doCheck = false;
@@ -52,6 +67,10 @@ stdenv.mkDerivation rec {
platforms = platforms.all;
maintainers = with maintainers; [ rafaelrc ];
mainProgram = "pkl";
+ sourceProvenance = with lib.sourceTypes; [
+ fromSource
+ binaryBytecode # mitm cache
+ ];
};
}
diff --git a/pkgs/by-name/pk/pkl/set_commit_id.patch b/pkgs/by-name/pk/pkl/set_commit_id.patch
new file mode 100644
index 000000000000..081a523c579a
--- /dev/null
+++ b/pkgs/by-name/pk/pkl/set_commit_id.patch
@@ -0,0 +1,20 @@
+diff --git a/buildSrc/src/main/kotlin/BuildInfo.kt b/buildSrc/src/main/kotlin/BuildInfo.kt
+index 0264375..dfabed1 100644
+--- a/buildSrc/src/main/kotlin/BuildInfo.kt
++++ b/buildSrc/src/main/kotlin/BuildInfo.kt
+@@ -87,14 +87,7 @@ open class BuildInfo(project: Project) {
+ val commitId: String by lazy {
+ // only run command once per build invocation
+ if (project === project.rootProject) {
+- val process = ProcessBuilder()
+- .command("git", "rev-parse", "--short", "HEAD")
+- .directory(project.rootDir)
+- .start()
+- process.waitFor().also { exitCode ->
+- if (exitCode == -1) throw RuntimeException(process.errorStream.reader().readText())
+- }
+- process.inputStream.reader().readText().trim()
++ "@commit_id@"
+ } else {
+ project.rootProject.extensions.getByType(BuildInfo::class.java).commitId
+ }You will also need to patch the binary so it points to the correct jre. The tests might require further workarounds to run though. |
Contributor
|
and after this patch it works fine diff --git a/pkgs/by-name/pk/pkl/package.nix b/pkgs/by-name/pk/pkl/package.nix
index 3c8a8a1e4e32..a871a1574759 100644
--- a/pkgs/by-name/pk/pkl/package.nix
+++ b/pkgs/by-name/pk/pkl/package.nix
@@ -3,7 +3,6 @@
, gradle
, lib
, fetchFromGitHub
-, jre
, git
}:
stdenv.mkDerivation rec {
@@ -34,10 +33,6 @@ stdenv.mkDerivation rec {
gradle
];
- buildInputs = [
- jre
- ];
-
mitmCache = gradle.fetchDeps {
inherit pname;
data = ./deps.json;
@@ -57,7 +52,9 @@ stdenv.mkDerivation rec {
installPhase = ''
mkdir -p "$out/bin"
- install -Dm755 "./pkl-cli/build/executable/jpkl" "$out/bin/pkl"
+ head -n2 ./pkl-cli/build/executable/jpkl | sed 's%java%${jdk17}/bin/java%' > "$out/bin/pkl"
+ tail -n+3 ./pkl-cli/build/executable/jpkl >> "$out/bin/pkl"
+ chmod 755 "$out/bin/pkl"
'';
meta = with lib; { |
Member
Author
|
@chayleaf Oh, I see you were also working on this, thanks! I managed to build it using the |
Contributor
|
set_commit_id.patch and postFetch is necessary because one of the tests checks that the commit ID is set. |
Member
Author
|
I noticed you added |
Contributor
|
Since the target jre version is 17 (see https://github.com/apple/pkl/blob/0.26.2/buildSrc/build.gradle.kts), it makes sense to build using jdk17 - that's the only reason. |
Contributor
|
oh yeah, just to clarify - the test checks that the language exports the commit ID of pkl, that's why it's necessary to set it (of course, the alternative is to leave .git and add git to native build inputs) |
Member
Author
|
@lilyball I started the native build work here: https://github.com/rafaelrc7/nixpkgs/blob/pkl/native/pkgs/by-name/pk/pkl/package.nix . If you want to help. Currently it is not building, I plan on creating a new PR after this one is merged. I believe the problems come from the fact that pkl seems to depend on an older version of graalvm (17). I changed a flag from |
This was referenced Jul 20, 2024
13 tasks
Contributor
|
I fixed the tests, but this requires #329554 (edit: it got merged) diffdiff --git a/pkgs/by-name/pk/pkl/disable_gradle_codegen_tests.patch b/pkgs/by-name/pk/pkl/disable_gradle_codegen_tests.patch
new file mode 100644
index 000000000000..b3e9d78970cf
--- /dev/null
+++ b/pkgs/by-name/pk/pkl/disable_gradle_codegen_tests.patch
@@ -0,0 +1,68 @@
+diff --git a/pkl-gradle/src/test/kotlin/org/pkl/gradle/JavaCodeGeneratorsTest.kt b/pkl-gradle/src/test/kotlin/org/pkl/gradle/JavaCodeGeneratorsTest.kt
+index 277d6c2..254477d 100644
+--- a/pkl-gradle/src/test/kotlin/org/pkl/gradle/JavaCodeGeneratorsTest.kt
++++ b/pkl-gradle/src/test/kotlin/org/pkl/gradle/JavaCodeGeneratorsTest.kt
+@@ -1,11 +1,13 @@
+ package org.pkl.gradle
+
+ import org.assertj.core.api.Assertions.assertThat
++import org.junit.jupiter.api.Disabled;
+ import org.junit.jupiter.api.Test
+ import kotlin.io.path.listDirectoryEntries
+ import kotlin.io.path.readText
+
+ class JavaCodeGeneratorsTest : AbstractTest() {
++ @Disabled("runTask doesn't use the MITM cache")
+ @Test
+ fun `generate code`() {
+ writeBuildFile()
+@@ -50,6 +52,7 @@ class JavaCodeGeneratorsTest : AbstractTest() {
+ )
+ }
+
++ @Disabled("runTask doesn't use the MITM cache")
+ @Test
+ fun `compile generated code`() {
+ writeBuildFile()
+@@ -66,6 +69,7 @@ class JavaCodeGeneratorsTest : AbstractTest() {
+ assertThat(addressClassFile).exists()
+ }
+
++ @Disabled("runTask doesn't use the MITM cache")
+ @Test
+ fun `no source modules`() {
+ writeFile(
+diff --git a/pkl-gradle/src/test/kotlin/org/pkl/gradle/KotlinCodeGeneratorsTest.kt b/pkl-gradle/src/test/kotlin/org/pkl/gradle/KotlinCodeGeneratorsTest.kt
+index 8ede4c6..93578b0 100644
+--- a/pkl-gradle/src/test/kotlin/org/pkl/gradle/KotlinCodeGeneratorsTest.kt
++++ b/pkl-gradle/src/test/kotlin/org/pkl/gradle/KotlinCodeGeneratorsTest.kt
+@@ -1,11 +1,13 @@
+ package org.pkl.gradle
+
+ import org.assertj.core.api.Assertions.assertThat
++import org.junit.jupiter.api.Disabled;
+ import org.junit.jupiter.api.Test
+ import kotlin.io.path.listDirectoryEntries
+ import kotlin.io.path.readText
+
+ class KotlinCodeGeneratorsTest : AbstractTest() {
++ @Disabled("runTask doesn't use the MITM cache")
+ @Test
+ fun `generate code`() {
+ writeBuildFile()
+@@ -51,6 +53,7 @@ class KotlinCodeGeneratorsTest : AbstractTest() {
+ )
+ }
+
++ @Disabled("runTask doesn't use the MITM cache")
+ @Test
+ fun `compile generated code`() {
+ writeBuildFile()
+@@ -66,6 +69,7 @@ class KotlinCodeGeneratorsTest : AbstractTest() {
+ assertThat(addressClassFile).exists()
+ }
+
++ @Disabled("runTask doesn't use the MITM cache")
+ @Test
+ fun `no source modules`() {
+ writeFile(
diff --git a/pkgs/by-name/pk/pkl/fix_kotlin_classpath.patch b/pkgs/by-name/pk/pkl/fix_kotlin_classpath.patch
new file mode 100644
index 000000000000..5b28c800adfd
--- /dev/null
+++ b/pkgs/by-name/pk/pkl/fix_kotlin_classpath.patch
@@ -0,0 +1,13 @@
+diff --git a/pkl-gradle/pkl-gradle.gradle.kts b/pkl-gradle/pkl-gradle.gradle.kts
+index 55a03fc..8968dff 100644
+--- a/pkl-gradle/pkl-gradle.gradle.kts
++++ b/pkl-gradle/pkl-gradle.gradle.kts
+@@ -40,7 +40,7 @@ sourceSets {
+ test {
+ // Remove Gradle distribution JARs from test compile classpath.
+ // This prevents a conflict between Gradle's and Pkl's Kotlin versions.
+- compileClasspath = compileClasspath.filter { !(it.path.contains("dists")) }
++ compileClasspath = compileClasspath.filter { !(it.path.contains("@gradle@") || it.path.contains("generated-gradle-jars/gradle-api-")) }
+ }
+ }
+
diff --git a/pkgs/by-name/pk/pkl/package.nix b/pkgs/by-name/pk/pkl/package.nix
index 273a7ec8f80b..d5fa99285c1c 100644
--- a/pkgs/by-name/pk/pkl/package.nix
+++ b/pkgs/by-name/pk/pkl/package.nix
@@ -1,5 +1,6 @@
{ stdenv
, lib
+, substituteAll
, fetchFromGitHub
, gradle
, jdk17
@@ -27,6 +28,14 @@ stdenv.mkDerivation rec {
'';
};
+ patches = [
+ (substituteAll {
+ src = ./fix_kotlin_classpath.patch;
+ gradle = gradle.unwrapped;
+ })
+ ./disable_gradle_codegen_tests.patch
+ ];
+
nativeBuildInputs = [ gradle ];
mitmCache = gradle.fetchDeps {
@@ -42,6 +51,8 @@ stdenv.mkDerivation rec {
"-Dorg.gradle.java.home=${jdk17}"
];
+ doCheck = true;
+
JAVA_TOOL_OPTIONS = "-Dfile.encoding=utf-8";
installPhase = '' |
13 tasks
Contributor
|
There's a new release of pkl with version 0.26.3. |
Contributor
|
Hi! Is there any update on this? Anything blocking this from going through? |
FliegendeWurst
requested changes
Oct 18, 2024
Member
FliegendeWurst
left a comment
FliegendeWurst
left a comment
There was a problem hiding this comment.
Does not build
FAILURE: Build failed with an exception.
* Where:
Build file '/build/source/buildSrc/build.gradle.kts' line: 3
* What went wrong:
Plugin [id: 'org.gradle.kotlin.kotlin-dsl', version: '4.5.0'] was not found in any of the following sources:
- Gradle Core Plugins (plugin is not in 'org.gradle' namespace)
- Included Builds (No included builds contain this plugin)
- Plugin Repositories (could not resolve plugin artifact 'org.gradle.kotlin.kotlin-dsl:org.gradle.kotlin.kotlin-dsl.gradle.plugin:4.5.0')
Searched in the following repositories:
MavenRepo
Gradle Central Plugin Repository
Contributor
|
I've tested this on NixOS 24.11 and the build and executable work great for me. |
Contributor
|
I've also tested this via Nix Shell on Darwin arm64. @FliegendeWurst can you provide more detail or reevaluate your previous review? This PR is a year old now. |
Member
|
I'll check again soon. Did you run with this change on master or on the PR branch? |
Member
|
Just checked again on NixOS 24.11 x64 Linux, same error. Note: I cherry-picked the PR onto nixos-24.11 to test. |
Member
|
Here is a working deps.json for 24.11. You need to rebase on master and run Details |
Contributor
|
Please also add tests as mentioned here #286658 (comment) |
bioball
reviewed
Feb 14, 2025
| gradleFlags = [ | ||
| "-x" "spotlessCheck" | ||
| "-DreleaseBuild=true" | ||
| "-Dorg.gradle.java.home=${jdk17}" |
There was a problem hiding this comment.
Heads up: Pkl 0.28 will require Java 21 to build.
Java 21 can be used to build Pkl 0.26.2 as well, so, you might want to just switch to it now.
bioball
reviewed
Feb 14, 2025
| - if (exitCode == -1) throw RuntimeException(process.errorStream.reader().readText()) | ||
| - } | ||
| - process.inputStream.reader().readText().trim() | ||
| + "@commit_id@" |
There was a problem hiding this comment.
You can get rid of this and replace it with -DcommitId=<commitId> once apple/pkl#954 is merged, and when building Pkl 0.28.
Member
|
have we got any news regarding this? |
Member
|
The package's |
Member
|
@chayleaf had you tried running your tests? i am trying to make it work in #402086 but am struggling |
Member
|
#402086 got merged. closing |
13 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
10 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of changes
A configuration as code language with rich validation and tooling.
Homepage: https://pkl-lang.org/
Source: https://github.com/apple/pkl/
Closes #286104 .
Things done
nix.conf? (See Nix manual)sandbox = relaxedsandbox = truenix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage./result/bin/)Add a 👍 reaction to pull requests you find important.