Kscript is broken on ArchLinux

[CLOVIS-AI]

Installed from the AUR.

Error message:

$ kscript 'println("Hello world")'
/usr/bin/kscript: line 49: /usr/share/kotlin/bin/kotlin: No such file or directory

Correct path to the Kotlin binary:

$ command -v kotlin
/usr/bin/kotlin

I don't really know what information could be of use, don't hesitate to ask for specific information you need.

[aartiPl]

Thanks for the report!

Can you please execute in the console:

1. echo $KOTLIN_HOME
2. KOTLIN_RUNNER=1 JAVACMD=echo kotlinc

[CLOVIS-AI]

$ echo $KOTLIN_HOME

$ KOTLIN_RUNNER=1 JAVACMD=echo kotlinc
-Xmx256M -Xms32M -Dkotlin.home=/usr/share/kotlin -cp /usr/share/kotlin/lib/kotlin-runner.jar org.jetbrains.kotlin.runner.Main

[aartiPl]

The second line is explaining why you get /usr/share/kotlin path for kotlin. (I am not exactly sure how this line work, but it is used to guess KOTLIN_HOME, if it is not set explicitly).
To make kscript working set your KOTLIN_HOME env variable to /usr , and it should start working. I will keep this issue open to allow other comments and suggestions.

[CLOVIS-AI]

It does not seem to fix the problem.

# .zshrc
export KOTLIN_HOME=/usr

$ echo $KOTLIN_HOME
/usr
$ kscript 'println("Hello world!")'
[kscript] [ERROR] Compilation of scriplet failed:
[kscript] [ERROR] Command     : bash -c /usr/share/kotlin/bin/kotlinc   -d '/home/ivan/.kscript/cache/jar_1075c8348f9943a25627037568233d8f/scriplet.jar' '/home/ivan/.kscript/cache/jar_1075c8348f9943a25627037568233d8f/Scriplet.kts' '/home/ivan/.kscript/cache/jar_1075c8348f9943a25627037568233d8f/Main_Scriplet.kt'
[kscript] [ERROR] Exit Code   : 127   
[kscript] [ERROR] Stdout      : 
[kscript] [ERROR] Stderr      : 
[kscript] [ERROR] bash: line 1: /usr/share/kotlin/bin/kotlinc: No such file or directory
[kscript] [ERROR]

I'm not sure why the error message is visually different yet complains about the same thing.

[aartiPl]

That's strange. Below you can see the code which is used to resolve kotlin home:

https://github.com/holgerbrandl/kscript/blob/master/src/main/kotlin/kscript/app/model/ConfigBuilder.kt

As you can see KOTLIN_HOME env goes first while resolving kotlinc.

Another relevant piece of code:

https://github.com/holgerbrandl/kscript/blob/master/src/kscript

But here also everything looks okay for me.

[sfesenko]

As you can see KOTLIN_HOME env goes first while resolving kotlinc.
But here also everything looks okay for me.

It's not ok, since kscript makes wrong assumption about location of kotlin binaries, and doesn't check their actual location.
In archlinux scripts are located in $PATH, not under $KOTLIN_HOME.

pacman -Ql kotlin | grep /bin/

kotlin /usr/bin/
kotlin /usr/bin/kotlin
kotlin /usr/bin/kotlinc
kotlin /usr/bin/kotlinc-js
kotlin /usr/bin/kotlinc-jvm

[aartiPl]

Is KOTLIN_HOME set by some installer e. g. SdkMan, or Kotlin installer? In other distros, KOTLIN_HOME is a separate directory that contains bin and lib. Here is a screenshot from Ubuntu:

So if it is different in archlinux, I guess there will be needed a patch for KScript to correct that behavior on that specific platform.

[CLOVIS-AI]

You can see the list of files here: https://archlinux.org/packages/community/any/kotlin/ (in the section ‘package contents’).

Basically it is just another system package with binaries in /usr/bin, JARS in /usr/share, etc.

[sfesenko]

1. everything fine with sdkman, this issue about Archlinux and kotlin package from official Archlinux repo (as mentioned by @CLOVIS-AI )
2. Assumption, that kotlin binaries are placed in $KOTLIN_HOME/bin is just wrong.
3. Implementation also doesn't look correct:
   1. GUESS_KOTLIN_HOME=$(KOTLIN_RUNNER=1 JAVACMD=echo kotlinc) ## here it assumes that kotlinc is present in $PATH
   2. KOTLIN_BIN="$KOTLIN_HOME/bin/" ## then it require, that kotlin is present in $KOTLIN_HOME/bin/ (without any checks)
   3. anyway, KOTLIN_HOME value will be ignored later, so nothing will work:

$ kscript
/home/sfesenko/.sdkman/candidates/kscript/current/bin/kscript: line 50: /usr/share/kotlin/bin/kotlin: No such file or directory

$ KOTLIN_HOME=/usr kscript
kscript - Enhanced scripting support for Kotlin on *nix-based systems.
...
Copyright : 2022 Holger Brandl
License : MIT
Version : v4.1.1
Website : https://github.com/holgerbrandl/kscript

$ KOTLIN_HOME=/usr kscript println
[kscript] [ERROR] Compilation of scriplet failed:
[kscript] [ERROR] Command : 'bash -c /usr/share/kotlin/bin/kotlinc -d '/home/sfesenko/.cache/kscript/jar_b58e7483b3914effc9ffa9c6ce4fb765/scriplet.jar' '/home/sfesenko/.cache/kscript/jar_b58e7483b3914effc9ffa9c6ce4fb765/Scriplet.kts' '/home/sfesenko/.cache/kscript/jar_b58e7483b3914effc9ffa9c6ce4fb765/Main_Scriplet.kt''
[kscript] [ERROR] Exit Code : 127
[kscript] [ERROR] Stdout : ''
[kscript] [ERROR] Stderr : 'bash: line 1: /usr/share/kotlin/bin/kotlinc: No such file or directory[nl]'
[kscript] [ERROR]

Since it already assumes, that kotlin/kotlinc are available in $PATH, all logic around $KOTLIN_HOME/bin/ may be removed and issue should be fixed.
Also I believe, that bash is not required to run kotlin/kotlinc (they don't have to be bash scripts)
As alternative option - kscript may embed kotlin/kotlinc logic, and just run java command, so it won't depend on bash / cmd/ kotlin / kotlinc / etc and won't run jvm multiple times (kscript will require either $KOTLIN_HOME or kotlin in $PATH to calculate $KOTLIN_HOME),

[aartiPl]

KScript is using KOTLIN_HOME in CommandResolver.kt in two places, so it is not enough to rely on PATH. I want to rework how scripting execution happens so that dependency on this environment variable might be relaxed.

BTW. Has KScript worked previously on ArchLinux?

[sfesenko]

4.0.0 is last version that works
Added dockerfile as test

this succeed

docker build -t kts --build-arg VERSION=4.0.0 .

but this failed

docker build -t kts --build-arg VERSION=4.1.0 .

Dockerfile

FROM archlinux

RUN pacman -Sy --noconfirm \
  which \
  jdk-openjdk \
  unzip \
  zip \
  kotlin 

RUN curl -s "https://get.sdkman.io" | bash

ARG VERSION=4.1.0
RUN . ~/.bashrc && sdk i kscript $VERSION

# Run test
RUN . ~/.bashrc; kscript 'println("Hi")'

ENTRYPOINT . ~/.bashrc; kscript 'println("Hi")'

[aartiPl]

Thanks a lot for checking! I plan to do one more big rework in kscript so that refactoring might resolve this issue. However, I will keep this issue open as it is an important use case.

[DareFox]

Tried to fix problem of ignored KOTLIN_HOME=/usr variable value, that @sfesenko mentioned. Turns out kotlinc automatically overwrites it with /usr/share/kotlin value

The only way I found to bypass this behavior is to change the name of KOTLIN_HOME in bash launch script to this:

src/kscript

## run it using command substitution to have just the user process once kscript is done
- COMMAND=$("${KOTLIN_BIN}kotlin" -classpath "${JAR_PATH}" kscript.app.KscriptKt "$OSTYPE" "$@")
+ COMMAND=$(export KOTLIN_HOME_BYPASS=$KOTLIN_HOME; "${KOTLIN_BIN}kotlin" -classpath "${JAR_PATH}" kscript.app.KscriptKt "$OSTYPE" "$@")

And read its value from ConfigBuilder
src/main/kotlin/../model/ConfigBuilder.kt

private fun resolveKotlinHome(osType: OsType): OsPath = path(
+   System.getenv("KOTLIN_HOME_BYPASS") ?:
    System.getenv("KOTLIN_HOME") ?:
    ShellUtils.guessKotlinHome(osType) ?:
    throw IllegalStateException("KOTLIN_HOME is not set and could not be inferred from context.")
)

This solved the problem with kscript finding kotlinc binary, but now I get a "NoClassDefFoundError" error because the stdlib JAR files are exist separately in /usr/share/kotlin/libs

Exception in thread "main" java.lang.NoClassDefFoundError: kotlin/script/templates/standard/ScriptTemplateWithArgs
        at java.base/java.lang.ClassLoader.defineClass1(Native Method)
        at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1017)
        at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174)
        at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555)
        at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458)
        at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452)
        at java.base/java.security.AccessController.doPrivileged(Native Method)
        at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
        at Main_Scriplet$Companion.main(Main_Scriplet.kt:5)
        at Main_Scriplet.main(Main_Scriplet.kt)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at org.jetbrains.kotlin.runner.AbstractRunner.run(runners.kt:70)
        at org.jetbrains.kotlin.runner.Main.run(Main.kt:188)
        at org.jetbrains.kotlin.runner.Main.main(Main.kt:198)
Caused by: java.lang.ClassNotFoundException: kotlin.script.templates.standard.ScriptTemplateWithArgs
        at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:476)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)

[DareFox]

Found fix, just create symlink to kotlinc in /usr/share/kotlin/bin

 sudo mkdir /usr/share/kotlin/bin
 sudo ln -s /usr/bin/kotlin /usr/share/kotlin/bin/kotlin   
 sudo ln -s /usr/bin/kotlinc /usr/share/kotlin/bin/kotlinc   

Maybe add this commands to PKGBUILD on Aur?

[aartiPl]

Found fix, just create symlink to kotlinc in /usr/share/kotlin/bin

 sudo mkdir /usr/share/kotlin/bin
 sudo ln -s /usr/bin/kotlin /usr/share/kotlin/bin/kotlin   
 sudo ln -s /usr/bin/kotlinc /usr/share/kotlin/bin/kotlinc   

@DareFox - this is a nice workaround - thanks for bringing it here.

Another workaround would be to install Kotlin using sdkman (alongside the system-installed one), as in kscript installation guide. That should also fix the problem, although I haven't tested it on ArchLinux.

[CLOVIS-AI]

Maybe add this commands to PKGBUILD on Aur?

Is it first-party? Who should be contacted to do it?

Another workaround would be to install Kotlin using sdkman

ArchLinux users are blessed with a good quality package manager, we tend to avoid installing software by other means for security/stability reasons.

[aartiPl]

@DareFox - I think those symbolic links should be added in the Kotlin package rather than in the KScript. This way, the behavior on the ArchLinux will be consistent with the Kotlin installation directory structure on other systems. And in fact, I think this is the best of all solutions discussed above.

Can you try to push the change to the maintainer of the Kotlin package on ArchLinux?

[aartiPl]

(at) HERE - let me discuss other solutions and why I don't think they are the best way to proceed. Let's assume we have the following structure on disk:

Additional Kotlin versions:
/opt/kotlin/1.7.20/* [bin/, lib/... directory]
/opt/kotlin/1.6.20/* [bin/, lib/... directory]
....
Default ArchLinux Kotlin version:
/usr/bin/kotlin
/usr/bin/kotlinc
/usr/share/kotlin/* [lib/*... directory]

1. Revert to the state of kscript as it was in version 4.0.0 and before (executing 'kotlinc'/'kotlin' without the absolute path and relying on the 'PATH' env variable)

In such a case, when we change 'KOTLIN_HOME' to one of the versions in '/opt', we will still have in 'PATH' binaries from the other version of Kotlin. So it may be wrong and will not work with libraries in 'KOTLIN_HOME'. We can, of course, change 'PATH' so that it will execute the correct version of 'kotlin' and 'kotlinc' scripts, but IMHO it's not really how "*_HOME" type variables work. They should direct to the whole, consistent environment connected with the Kotlin version.

2. Use Kotlin compiler jar directly without executing shell scripts

This way, we can eliminate dependency on 'kotlin'/'kotlinc', but we do not have any guarantees that it will not break in new versions of Kotlin: calling jar files directly, without using scripts, is possible (and it works - I have tested it), but it is not a public API of Kotlin, and it can break at any time in a new version of Kotlin.

3. In the 'kscript' installation package, add symbolic links between Kotlin executables '/usr/bin/' and 'KOTLIN_HOME/bin/'

It's not a 'kscript' thing to provide those links. The best solution is to provide links in the Kotlin package. It is consistent with the way Kotlin is distributed.

To summarize, I am reluctant to change 'kscript' to fix this problem because it seems that the issue should be resolved in the Kotlin package in ArchLinux.

If anyone can contact the Kotlin package maintainer for ArchLinux, and ask to add symbolic links between 'KOTLIN_HOME/bin/' and '/usr/bin/', that would be the best solution.

[aartiPl]

I have added information about the problem in the README file with a link to this ticket.

[aartiPl]

@ here - I need some help with releasing to ArchLinux. Please have a look at:
#376 (comment)

If you know how to solve that problem, please comment:
#376