Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

<worker> section to enable a set of configuration in a specified worker #1392

Closed
tagomoris opened this issue Dec 27, 2016 · 1 comment
Closed
Labels
feature request *Deprecated Label* Use enhancement label in general v0.14

Comments

@tagomoris
Copy link
Member

tagomoris commented Dec 27, 2016

Symmetric multi worker processing #1386 doesn't permit to execute any plugins and configurations in just one process.
But there are requirements and feature requests for such use cases. For example, executing plugins for monitoring (should be unique in a server).

This is an idea to keep symmetric semantics for multi process workers, but to realize such workloads.

  • add <worker N> section in configuration top-level
  • enable that section only in the specified worker (specified by worker id)
  • allow to run plugins only in <worker> sections, which doesn't support multi worker feature
<system>
  workers 8
  root_dir /path/fluentd/root
</system>
<source>  # top-level sections works on all workers in parallel
  @id edge_input
  @type forward
  @label @traffic
  port 24224
</source>
<label @traffic>  # this section works on all workers too
  <match **>
    @id backend_output
    @type forward
    <buffer>
      @type file
      flush_interval 8m
    </buffer>
  </match>
</label>
<worker 0> # this section works only on first worker process
  <source>
    @id system_log_reader
    @type tail
    tag monitoring
    path /path/now/watching/...
  </source>
  <match monitoring>
    @id output_for_monitoring
    @type elasticsearch
    # ...
  </match>
</worker>
@tagomoris tagomoris added feature request *Deprecated Label* Use enhancement label in general v0.14 labels Dec 27, 2016
@tagomoris
Copy link
Member Author

Points to be considered:

  • configuration dump is now done in worker_id=0 under a premise that "all workers are working with just same configuration" - this feature should update logging.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request *Deprecated Label* Use enhancement label in general v0.14
Projects
None yet
Development

No branches or pull requests

1 participant