Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

'Quick Start' doesn't work. #1310

Closed
suzukaze opened this issue Nov 11, 2016 · 1 comment · Fixed by #1313
Closed

'Quick Start' doesn't work. #1310

suzukaze opened this issue Nov 11, 2016 · 1 comment · Fixed by #1313
Labels
bug Something isn't working

Comments

@suzukaze
Copy link
Member

Environment

OS: macOS Sierra version 10.12.1
ruby version: ruby 2.3.0p0 (2015-12-25 revision 53290) [x86_64-darwin15]
fluentd version: fluentd-0.14.8

How to reproduction

I executed the process of 'Quick Start' in README.md.

$ gem install fluentd
$ fluentd -s conf

fluentd -s conf generates conf/fluent.conf.
fluent.conf is:

# In v1 configuration, type and id are @ prefix parameters.
# @type and @id are recommended. type and id are still available for backward compatibility

## built-in TCP input
## $ echo <json> | fluent-cat <tag>
<source>
  @type forward
  @id forward_input
</source>

## built-in UNIX socket input
#<source>
#  @type unix
#</source>

# HTTP input
# http://localhost:8888/<tag>?json=<json>
<source>
  @type http
  @id http_input

  port 8888
</source>

## File input
## read apache logs with tag=apache.access
#<source>
#  @type tail
#  format apache
#  path /var/log/httpd-access.log
#  tag apache.access
#</source>

## Mutating event filter
## Add hostname and tag fields to apache.access tag events
#<filter apache.access>
#  @type record_transformer
#  <record>
#    hostname ${hostname}
#    tag ${tag}
#  </record>
#</filter>

## Selecting event filter
## Remove unnecessary events from apache prefixed tag events
#<filter apache.**>
#  @type grep
#  include1 method GET # pass only GET in 'method' field
#  exclude1 message debug # remove debug event
#</filter>

# Listen HTTP for monitoring
# http://localhost:24220/api/plugins
# http://localhost:24220/api/plugins?type=TYPE
# http://localhost:24220/api/plugins?tag=MYTAG
<source>
  @type monitor_agent
  @id monitor_agent_input

  port 24220
</source>

# Listen DRb for debug
<source>
  @type debug_agent
  @id debug_agent_input

  bind 127.0.0.1
  port 24230
</source>

## match tag=apache.access and write to file
#<match apache.access>
#  @type file
#  path /var/log/fluent/access
#</match>

## match tag=debug.** and dump to console
<match debug.**>
  @type stdout
  @id stdout_output
</match>

# match tag=system.** and forward to another fluent server
<match system.**>
  @type forward
  @id forward_output

  <server>
    host 192.168.0.11
  </server>
  <secondary>
    <server>
      host 192.168.0.12
    </server>
  </secondary>
</match>

## match tag=myapp.** and forward and write to file
#<match myapp.**>
#  @type copy
#  <store>
#    @type forward
#    buffer_type file
#    buffer_path /var/log/fluent/myapp-forward
#    retry_limit 50
#    flush_interval 10s
#    <server>
#      host 192.168.0.13
#    </server>
#  </store>
#  <store>
#    @type file
#    path /var/log/fluent/myapp
#  </store>
#</match>

## match fluent's internal events
#<match fluent.**>
#  @type null
#</match>

## match not matched logs and write to file
#<match **>
#  @type file
#  path /var/log/fluent/else
#  compress gz
#</match>

## Label: For handling complex event routing
#<label @STAGING>
#  <match system.**>
#    @type forward
#    @id staging_forward_output
#    <server>
#      host 192.168.0.101
#    </server>
#  </match>
#</label>

Error

Error logs are:

$ 2016-11-11 18:46:07 +0900 [info]: reading config file path="conf/fluent.conf"
2016-11-11 18:46:07 +0900 [info]: starting fluentd-0.14.8
2016-11-11 18:46:07 +0900 [info]: spawn command to main: /Users/hiroejun/.rbenv/versions/2.3.0/bin/ruby -Eascii-8bit:ascii-8bit /Users/hiroejun/.rbenv/versions/2.3.0/bin/fluentd -c conf/fluent.conf --under-supervisor
2016-11-11 18:46:13 +0900 [info]: reading config file path="conf/fluent.conf"
2016-11-11 18:46:13 +0900 [info]: starting fluentd-0.14.8 without supervision
2016-11-11 18:46:13 +0900 [info]: gem 'fluentd' version '0.14.8'
2016-11-11 18:46:13 +0900 [info]: adding match pattern="debug.**" type="stdout"
2016-11-11 18:46:13 +0900 [info]: adding match pattern="system.**" type="forward"
2016-11-11 18:46:13 +0900 [info]: adding forwarding server '192.168.0.12:24224' host="192.168.0.12" port=24224 weight=60 plugin_id="object:3fc7ead4663c"
2016-11-11 18:46:13 +0900 [error]: unexpected error error="undefined method `compress' for nil:NilClass"
2016-11-11 18:46:13 +0900 [error]: /Users/hiroejun/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/fluentd-0.14.8/lib/fluent/plugin/out_forward.rb:145:in `configure'
2016-11-11 18:46:13 +0900 [error]: /Users/hiroejun/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/fluentd-0.14.8/lib/fluent/plugin/output.rb:319:in `configure'
2016-11-11 18:46:13 +0900 [error]: /Users/hiroejun/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/fluentd-0.14.8/lib/fluent/plugin_helper/inject.rb:95:in `configure'
2016-11-11 18:46:13 +0900 [error]: /Users/hiroejun/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/fluentd-0.14.8/lib/fluent/plugin_helper/event_emitter.rb:60:in `configure'
2016-11-11 18:46:13 +0900 [error]: /Users/hiroejun/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/fluentd-0.14.8/lib/fluent/compat/output.rb:482:in `configure'
2016-11-11 18:46:13 +0900 [error]: /Users/hiroejun/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/fluentd-0.14.8/lib/fluent/plugin/out_forward.rb:120:in `configure'
2016-11-11 18:46:13 +0900 [error]: /Users/hiroejun/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/fluentd-0.14.8/lib/fluent/agent.rb:127:in `add_match'
2016-11-11 18:46:13 +0900 [error]: /Users/hiroejun/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/fluentd-0.14.8/lib/fluent/agent.rb:70:in `block in configure'
2016-11-11 18:46:13 +0900 [error]: /Users/hiroejun/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/fluentd-0.14.8/lib/fluent/agent.rb:63:in `each'
2016-11-11 18:46:13 +0900 [error]: /Users/hiroejun/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/fluentd-0.14.8/lib/fluent/agent.rb:63:in `configure'
2016-11-11 18:46:13 +0900 [error]: /Users/hiroejun/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/fluentd-0.14.8/lib/fluent/root_agent.rb:86:in `configure'
2016-11-11 18:46:13 +0900 [error]: /Users/hiroejun/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/fluentd-0.14.8/lib/fluent/engine.rb:119:in `configure'
2016-11-11 18:46:13 +0900 [error]: /Users/hiroejun/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/fluentd-0.14.8/lib/fluent/engine.rb:93:in `run_configure'
2016-11-11 18:46:13 +0900 [error]: /Users/hiroejun/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/fluentd-0.14.8/lib/fluent/supervisor.rb:693:in `run_configure'
2016-11-11 18:46:13 +0900 [error]: /Users/hiroejun/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/fluentd-0.14.8/lib/fluent/supervisor.rb:453:in `block in run_worker'
2016-11-11 18:46:13 +0900 [error]: /Users/hiroejun/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/fluentd-0.14.8/lib/fluent/supervisor.rb:626:in `main_process'
2016-11-11 18:46:13 +0900 [error]: /Users/hiroejun/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/fluentd-0.14.8/lib/fluent/supervisor.rb:449:in `run_worker'
2016-11-11 18:46:13 +0900 [error]: /Users/hiroejun/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/fluentd-0.14.8/lib/fluent/command/fluentd.rb:288:in `<top (required)>'
2016-11-11 18:46:13 +0900 [error]: /Users/hiroejun/.rbenv/versions/2.3.0/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in `require'
2016-11-11 18:46:13 +0900 [error]: /Users/hiroejun/.rbenv/versions/2.3.0/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in `require'
2016-11-11 18:46:13 +0900 [error]: /Users/hiroejun/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/fluentd-0.14.8/bin/fluentd:5:in `<top (required)>'
2016-11-11 18:46:13 +0900 [error]: /Users/hiroejun/.rbenv/versions/2.3.0/bin/fluentd:23:in `load'
2016-11-11 18:46:13 +0900 [error]: /Users/hiroejun/.rbenv/versions/2.3.0/bin/fluentd:23:in `<main>'
2016-11-11 18:46:13 +0900 [info]: Worker 0 finished unexpectedly with status 1

Solution Plan

Quick Start works when the configure bellow is cmmented.

# match tag=system.** and forward to another fluent server
<match system.**>
  @type forward
  @id forward_output

  <server>
    host 192.168.0.11
  </server>
  <secondary>
    <server>
      host 192.168.0.12
    </server>
  </secondary>
</match>

Should we comment this configure('match tag=system.** and forward to another fluent server') by default?

@tagomoris
Copy link
Member

I identified the root cause of this problem: out_forward has a bug to crash when it's used as secondary plugin.
I pushed a fix for this problem #1313.
@suzukaze Thank you for reporting!

@tagomoris tagomoris added the bug Something isn't working label Nov 15, 2016
@tagomoris tagomoris changed the title 'Qick Start' doesn't work. 'Quick Start' doesn't work. Nov 15, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants