Skip to content

prepare-offline-pack does not work with the logstash-filter-translate plugin #15102

@spinenkoia

Description

@spinenkoia

Logstash information:

  1. Logstash version - 8.8.0
  2. Logstash installation source - docker image logstash:8.8.0
  3. How is Logstash being run - docker

Plugins installed: logstash-filter-translate

JVM (e.g. java -version):
openjdk 17.0.7 2023-04-18
OpenJDK Runtime Environment Temurin-17.0.7+7 (build 17.0.7+7)
OpenJDK 64-Bit Server VM Temurin-17.0.7+7 (build 17.0.7+7, mixed mode, sharing)

OS version (uname -a if on a Unix-like system):
Linux 563ff1950c4e 5.10.104-linuxkit #1 SMP Thu Mar 17 17:08:06 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Description of the problem including expected versus actual behavior:
When installing the logstash-filter-translate plugin using prepare-offline-pack, the transitive library is lost
Since version 3.4.1, logstash-filter-translate has become dependent on psych 5.1.0, which in turn depends on snakeyaml 2.6. When packing prepare-offline-package, the gem file is not included snakeyaml-engine-2.6.jar . When installing a plugin from such an archive, dependencies break

Steps to reproduce:

  1. Run Logstash in docker
    docker run --rm -ti logstash:8.8.0 bash

  2. Install plugin
    bin/logstash-plugin install --version 3.4.2 logstash-filter-translate

  3. Create archive
    bin/logstash-plugin prepare-offline-pack --overwrite logstash-filter-translate

  4. Copy archive
    docker cp 94c7a08b5d88:/usr/share/logstash/logstash-offline-plugins-8.8.0.zip .
    where 94c7a08b5d88 is the container ID (can be viewed via docker ps)

  5. Exit from container
    ctrl+D

  6. Create new container
    docker run --rm -ti logstash:8.8.0 bash

  7. Copy archive to container
    docker cp logstash-offline-plugins-8.8.0.zip 563ff1950c4e:/

  8. Install plugin from offline pack
    bin/logstash-plugin install file:///logstash-offline-plugins-8.8.0.zip

  9. Try to see the list of plugins
    bin/logstash-plugin list
    we see an error: no such file to load -- org/snakeyaml/snakeyaml-engine/2.6/snakeyaml-engine-2.6.jar (LoadError)

Provide logs (if relevant):
logstash@563ff1950c4e:~$ bin/logstash-plugin list
Using bundled JDK: /usr/share/logstash/jdk
RuntimeError:

you might need to reinstall the gem which depends on the missing jar or in case there is Jars.lock then resolve the jars with `lock_jars` command

no such file to load -- org/snakeyaml/snakeyaml-engine/2.6/snakeyaml-engine-2.6.jar (LoadError)
do_require at /usr/share/logstash/vendor/jruby/lib/ruby/stdlib/jar_dependencies.rb:356
require_jar at /usr/share/logstash/vendor/jruby/lib/ruby/stdlib/jar_dependencies.rb:265
require_jar_with_block at /usr/share/logstash/vendor/jruby/lib/ruby/stdlib/jar_dependencies.rb:307
require_jar at /usr/share/logstash/vendor/jruby/lib/ruby/stdlib/jar_dependencies.rb:264
require_jar at /usr/share/logstash/vendor/jruby/lib/ruby/stdlib/jar_dependencies.rb:363

at /usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/psych-5.1.0-java/lib/psych_jars.rb:5
require at org/jruby/RubyKernel.java:1017
require_relative at org/jruby/RubyKernel.java:1045
at /usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/psych-5.1.0-java/lib/psych.rb:5
require at org/jruby/RubyKernel.java:1017
require at /usr/share/logstash/vendor/jruby/lib/ruby/stdlib/rubygems/core_ext/kernel_require.rb:85
at /usr/share/logstash/vendor/jruby/lib/ruby/stdlib/yaml.rb:4
require at org/jruby/RubyKernel.java:1017
require at /usr/share/logstash/vendor/jruby/lib/ruby/stdlib/rubygems/core_ext/kernel_require.rb:85
at /usr/share/logstash/lib/pluginmanager/util.rb:19
require at org/jruby/RubyKernel.java:1017
require at /usr/share/logstash/vendor/jruby/lib/ruby/stdlib/rubygems/core_ext/kernel_require.rb:85
at /usr/share/logstash/lib/pluginmanager/main.rb:31
LoadError: no such file to load -- org/snakeyaml/snakeyaml-engine/2.6/snakeyaml-engine-2.6.jar
require at org/jruby/RubyKernel.java:1017
require at /usr/share/logstash/vendor/jruby/lib/ruby/stdlib/rubygems/core_ext/kernel_require.rb:85
do_require at /usr/share/logstash/vendor/jruby/lib/ruby/stdlib/jar_dependencies.rb:353
require_jar at /usr/share/logstash/vendor/jruby/lib/ruby/stdlib/jar_dependencies.rb:265
require_jar_with_block at /usr/share/logstash/vendor/jruby/lib/ruby/stdlib/jar_dependencies.rb:307
require_jar at /usr/share/logstash/vendor/jruby/lib/ruby/stdlib/jar_dependencies.rb:264
require_jar at /usr/share/logstash/vendor/jruby/lib/ruby/stdlib/jar_dependencies.rb:363
at /usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/psych-5.1.0-java/lib/psych_jars.rb:5
require at org/jruby/RubyKernel.java:1017
require_relative at org/jruby/RubyKernel.java:1045
at /usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/psych-5.1.0-java/lib/psych.rb:5
require at org/jruby/RubyKernel.java:1017
require at /usr/share/logstash/vendor/jruby/lib/ruby/stdlib/rubygems/core_ext/kernel_require.rb:85
at /usr/share/logstash/vendor/jruby/lib/ruby/stdlib/yaml.rb:4
require at org/jruby/RubyKernel.java:1017
require at /usr/share/logstash/vendor/jruby/lib/ruby/stdlib/rubygems/core_ext/kernel_require.rb:85
at /usr/share/logstash/lib/pluginmanager/util.rb:19
require at org/jruby/RubyKernel.java:1017
require at /usr/share/logstash/vendor/jruby/lib/ruby/stdlib/rubygems/core_ext/kernel_require.rb:85
at /usr/share/logstash/lib/pluginmanager/main.rb:31

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions