Skip to content

Manage template (ILM) fails if settings section is missing #1040

@hackery

Description

@hackery

Issue elastic/logstash#11893 was opened by @tmanninger but I think needs to be in this repo instead.
I've done some rewording and provided details more relevant to our environment.

Logstash information:

  1. Logstash version 7.4.0
  2. Logstash installation source: RPM package from release repo
  3. How is Logstash being run: systemd
  4. How was the Logstash Plugin installed: bundled

JVM :

  1. JVM version (java -version)
openjdk version "11.0.11" 2021-04-20 LTS
OpenJDK Runtime Environment 18.9 (build 11.0.11+9-LTS)
OpenJDK 64-Bit Server VM 18.9 (build 11.0.11+9-LTS, mixed mode, sharing)
  1. JVM installation source: RPM package from release repo
  2. Value of the JAVA_HOME environment variable: not set

OS version :

Linux xxx-logstash-staging 3.10.0-1160.31.1.el7.x86_64 #1 SMP Thu Jun 10 13:32:12 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Description of the problem including expected versus actual behavior:

If the plugin includes configuration for ILM, adding these settings to the template fails if the template has no "settings" section. The log file shows an opaque NoMethodError message from add_ilm_settings_to_template:

Failed to install template. {:message=>"undefined method `update' for nil:NilClass", :class=>"NoMethodError", :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.3.1-java/lib/logstash/outputs/elasticsearch/template_manager.rb:39:in `add_ilm_settings_to_template'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.3.1-java/lib/logstash/outputs/elasticsearch/template_manager.rb:14:in `install_template'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.3.1-java/lib/logstash/outputs/elasticsearch/common.rb:197:in `install_template'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.3.1-java/lib/logstash/outputs/elasticsearch/common.rb:53:in `block in setup_after_successful_connection'"]}

Expected: Logstash would apply ILM settings even when a minimal template has no static settings.
Actual: it doesn't. Further, the pipeline loads and runs, obscuring the error.

template_manager.rb should create the settings field if it does not exist, before attempting to populate it.

Further, the error message should provide the context of which pipeline and template were being processed - I encountered this while adding ILM to a system which already had many pipelines, and had to do a fair bit of digging to unearth the problem.

Workaround: add an empty settings field to all templates to be used with ILM.

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions