Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Can not start fluentd service #527

Closed
chenjienn opened this issue Jan 17, 2019 · 9 comments
Closed

Can not start fluentd service #527

chenjienn opened this issue Jan 17, 2019 · 9 comments
Labels
question

Comments

@chenjienn
Copy link

chenjienn commented Jan 17, 2019 •
edited by cosmo0920
Loading

Problem

I try starting fluentd service using command 'systemctl start td-agent'
but it always result in:
Job for td-agent.service failed because the control process exited with error code. See "systemctl status td-agent.service" and "journalctl -xe" for details.

and the fluentd log is below:

2019-01-17 21:30:06 +0800 [info]: parsing config file is succeeded path="/etc/td-agent/td-agent.conf"
2019-01-17 21:30:06 +0800 [warn]: [output_td] secondary type should be same with primary one primary="Fluent::Plugin::TreasureDataLogOutput" secondary="Fluent::Plugin::FileOutput"
2019-01-17 21:30:06 +0800 [warn]: Consider to specify log_level with @log_level.
2019-01-17 21:30:06 +0800 [info]: HEAD http://elastic:********@10.23.33.35:19200/ [status:200, request:0.003s, query:n/a]
2019-01-17 21:30:06 +0800 [info]: Connection opened to Elasticsearch cluster => {:host=>"10.23.33.35", :port=>19200, :scheme=>"http", :user=>"elastic", :password=>"obfuscated"}, {:host=>"10.23.33.36", :port=>19200, :scheme=>"http", :user=>"elastic", :password=>"obfuscated"}
2019-01-17 21:30:06 +0800 [info]: GET http://elastic:********@10.23.33.36:19200/ [status:200, request:0.003s, query:n/a]
2019-01-17 21:30:06 +0800 [info]: Detected ES 6.x: ES 7.x will only accept `_doc` in type_name.
2019-01-17 21:30:07 +0800 [info]: parsing config file is succeeded path="/etc/td-agent/td-agent.conf"
2019-01-17 21:30:07 +0800 [warn]: [output_td] secondary type should be same with primary one primary="Fluent::Plugin::TreasureDataLogOutput" secondary="Fluent::Plugin::FileOutput"
2019-01-17 21:30:07 +0800 [warn]: Consider to specify log_level with @log_level.
2019-01-17 21:30:07 +0800 [info]: HEAD http://elastic:***@10.23.33.35:19200/ [status:200, request:0.003s, query:n/a]
2019-01-17 21:30:07 +0800 [info]: Connection opened to Elasticsearch cluster => {:host=>"10.23.33.35", :port=>19200, :scheme=>"http", :user=>"elastic", :password=>"obfuscated"}, {:host=>"10.23.33.36", :port=>19200, :scheme=>"http", :user=>"elastic", :password=>"obfuscated"}
2019-01-17 21:30:07 +0800 [info]: GET http://elastic:***@10.23.33.36:19200/ [status:200, request:0.004s, query:n/a]
2019-01-17 21:30:07 +0800 [info]: Detected ES 6.x: ES 7.x will only accept `_doc` in type_name.

it seems have no error, but why it can not startup ?

Using Fluentd and ES plugin versions

  • td-agent 3.2.1
    • paste result of fluentd --version or td-agent --version
  • ES plugin (how to check the plugin version)
  • ES version 6.2.3
@chenjienn
Copy link
Author

td-agent.txt
Here is my config

@cosmo0920
Copy link
Collaborator

Could you share your journalctl log?
It seems that Fluent log says that Fluentd starts and runs normally.

@chenjienn
Copy link
Author

chenjienn commented Jan 17, 2019 •
edited by cosmo0920
Loading

Could you share your journalctl log?
It seems that Fluent log says that Fluentd starts and runs normally.

Jan 17 21:58:46 CaihCtPrvLogstash002 systemd[1]: Starting td-agent: Fluentd based data collector for Treasure Data...
-- Subject: Unit td-agent.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit td-agent.service has begun starting up.
Jan 17 21:58:47 CaihCtPrvLogstash002 fluentd[104945]: /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.6/lib/fluent/config/section.rb:178:in `block in generate': parse: init is 
Jan 17 21:58:47 CaihCtPrvLogstash002 fluentd[104945]: from /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.6/lib/fluent/config/section.rb:169:in `each'
Jan 17 21:58:47 CaihCtPrvLogstash002 fluentd[104945]: from /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.6/lib/fluent/config/section.rb:169:in `generate'
Jan 17 21:58:47 CaihCtPrvLogstash002 fluentd[104945]: from /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.6/lib/fluent/configurable.rb:94:in `configure'
Jan 17 21:58:47 CaihCtPrvLogstash002 fluentd[104945]: from /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.6/lib/fluent/plugin/base.rb:57:in `configure'
Jan 17 21:58:47 CaihCtPrvLogstash002 fluentd[104945]: from /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.6/lib/fluent/plugin_id.rb:39:in `configure'
Jan 17 21:58:47 CaihCtPrvLogstash002 fluentd[104945]: from /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.6/lib/fluent/log.rb:580:in `configure'
Jan 17 21:58:47 CaihCtPrvLogstash002 fluentd[104945]: from /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.6/lib/fluent/plugin_helper/event_emitter.rb:73:in `configure'
Jan 17 21:58:47 CaihCtPrvLogstash002 fluentd[104945]: from /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.6/lib/fluent/plugin_helper/extract.rb:78:in `configure'
Jan 17 21:58:47 CaihCtPrvLogstash002 fluentd[104945]: from /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.6/lib/fluent/plugin_helper/parser.rb:83:in `configure'
Jan 17 21:58:47 CaihCtPrvLogstash002 fluentd[104945]: from /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.6/lib/fluent/plugin_helper/server.rb:312:in `configure'
Jan 17 21:58:47 CaihCtPrvLogstash002 fluentd[104945]: from /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.6/lib/fluent/plugin/in_tcp.rb:44:in `configure'
Jan 17 21:58:47 CaihCtPrvLogstash002 fluentd[104945]: from /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.6/lib/fluent/plugin.rb:164:in `configure'
Jan 17 21:58:47 CaihCtPrvLogstash002 fluentd[104945]: from /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.6/lib/fluent/root_agent.rb:282:in `add_source'
Jan 17 21:58:47 CaihCtPrvLogstash002 fluentd[104945]: from /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.6/lib/fluent/root_agent.rb:122:in `block in configure'
Jan 17 21:58:47 CaihCtPrvLogstash002 fluentd[104945]: from /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.6/lib/fluent/root_agent.rb:118:in `each'
Jan 17 21:58:47 CaihCtPrvLogstash002 fluentd[104945]: from /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.6/lib/fluent/root_agent.rb:118:in `configure'
Jan 17 21:58:47 CaihCtPrvLogstash002 fluentd[104945]: from /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.6/lib/fluent/engine.rb:131:in `configure'
Jan 17 21:58:47 CaihCtPrvLogstash002 fluentd[104945]: from /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.6/lib/fluent/engine.rb:96:in `run_configure'
Jan 17 21:58:47 CaihCtPrvLogstash002 fluentd[104945]: from /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.6/lib/fluent/supervisor.rb:795:in `run_configure'
Jan 17 21:58:47 CaihCtPrvLogstash002 fluentd[104945]: from /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.6/lib/fluent/supervisor.rb:579:in `dry_run'
Jan 17 21:58:47 CaihCtPrvLogstash002 fluentd[104945]: from /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.6/lib/fluent/supervisor.rb:597:in `supervise'
Jan 17 21:58:47 CaihCtPrvLogstash002 fluentd[104945]: from /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.6/lib/fluent/supervisor.rb:502:in `run_supervisor'
Jan 17 21:58:47 CaihCtPrvLogstash002 fluentd[104945]: from /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.6/lib/fluent/command/fluentd.rb:310:in `<top (required)>'
Jan 17 21:58:47 CaihCtPrvLogstash002 fluentd[104945]: from /opt/td-agent/embedded/lib/ruby/site_ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:in `require'
Jan 17 21:58:47 CaihCtPrvLogstash002 fluentd[104945]: from /opt/td-agent/embedded/lib/ruby/site_ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:in `require'
Jan 17 21:58:47 CaihCtPrvLogstash002 fluentd[104945]: from /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.6/bin/fluentd:8:in `<top (required)>'
Jan 17 21:58:47 CaihCtPrvLogstash002 fluentd[104945]: from /opt/td-agent/embedded/bin/fluentd:23:in `load'
Jan 17 21:58:47 CaihCtPrvLogstash002 fluentd[104945]: from /opt/td-agent/embedded/bin/fluentd:23:in `<main>'
Jan 17 21:58:47 CaihCtPrvLogstash002 systemd[1]: td-agent.service: control process exited, code=exited status=1
Jan 17 21:58:47 CaihCtPrvLogstash002 systemd[1]: Failed to start td-agent: Fluentd based data collector for Treasure Data.
-- Subject: Unit td-agent.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit td-agent.service has failed.
-- 
-- The result is failed.
Jan 17 21:58:47 CaihCtPrvLogstash002 systemd[1]: Unit td-agent.service entered failed state.
Jan 17 21:58:47 CaihCtPrvLogstash002 systemd[1]: td-agent.service failed.

@cosmo0920
Copy link
Collaborator

Hmm..., I'll try to reproduce this issue later.

@chenjienn
Copy link
Author

Hmm..., I'll try to reproduce this issue later.

ok , actually i was testing logstash against fluentd to compare their performence when transport message to elasticsearch , if you can solve it as soon as possible that would be grateful.

@cosmo0920
Copy link
Collaborator

This is caused by lack of <parse> section in in_tcp configuration:

<source>
  @type tcp
  tag filebeat.events
  port 2333
  bind 0.0.0.0
  # parse section is needed in Fluentd v1
  <parse>
     @type your_parser_type
  </parse>
</source>

@chenjienn
Copy link
Author

This is caused by lack of <parse> section in in_tcp configuration:

<source>
  @type tcp
  tag filebeat.events
  port 2333
  bind 0.0.0.0
  # parse section is needed in Fluentd v1
  <parse>
     @type your_parser_type
  </parse>
</source>

thanks!

@cosmo0920 cosmo0920 mentioned this issue Jan 18, 2019
Closed
@cosmo0920
Copy link
Collaborator

I also reported fluent/fluentd#2266 and send PR fluent/fluentd#2267 about this issue.
This ambigious error message due to Fluentd-core.

@arangyupgrade
Copy link

I'm having the same issue since yesterday!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
question
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@cosmo0920 @chenjienn @arangyupgrade