yaml config

[ganmacs]

Signed-off-by: Yuta Iwama [email protected]

Which issue(s) this PR fixes:
ref: #554

What this PR does / why we need it:

I tried to create the feature of the config file in yaml .
I don't intend to replace this config format with the current one.

What do you think? @repeatedly @cosmo0920 @ashie

• !fluent/json which is a yaml tag means that this value parser as json object

• !fluent/s which is a yaml tag means that this value parser as ruby's string

match: 
  key1: "ENV['ROOT']"
  key2: !fluent/s "ENV['ROOT']"

means

<match>
  key ENV['ROOT']
  key "ENV['ROOT']"
</match>

• !include which is a yaml tag is equivalent to @include in the current config.

• use $ instead of @ because @ is reserved word in yaml

• worker example

worker:
  $arg: 1 
  config:
    - source:
    ....

means

<worker 1>
  <source>
    ....
  </source>
</worker>

• label example

label:
  $name: '@test'
  config:
    - match:
       ....

means

<label @test>
  <match>
    ....
  </match>
</label>

• source example

source:
  $type: dummy

means

<source>
  @type dummy
</source>

• filter and match example

filter:
  $tag: tag
  $type:  grep
  ...
match:
  $tag: tag
  $type: stdout
  buffer:
    $type: memory

means

<filter tag>
  @type grep
   ...
</filter>

<match tag>
  @type stdout
  <buffer>
    @type memory
  </buffer>
</match>

Example

system:
  workers: 2

config:
  - worker:
      $arg: 1
      config:
        - source:
            $type: dummy
            dummy: !fluent/json { "message": "hello" }
            tag: dummy1.t
        - filter:
            $type: record_transformer
            $tag: dummy1.*
            record:
              - hostname: !fluent/s "#{Socket.gethostname}"
                tag: ${tag}
        - match:
            $type: stdout
            $tag: dummy1.*
            buffer:
              $type: memory
              flush_interval: 1s
  - worker:
      $arg: 1
      config:
        - source:
            $type: dummy
            $label: '@test'
            tag: test1
        - source:
            $type: dummy
            $label: '@test'
            tag: test2
        - label:
            $name: '@test'
            config:
              - match:
                  $tag: test2
                  $type: stdout
                  buffer:
                    $type: memory
                    flush_interval: 1s
              - match:
                  $tag: test1
                  $type: stdout
                  buffer:
                    $type: memory
                    flush_interval: 1s
  - source:
      $type: dummy
      tag: tag
  - match:
      $tag: tag
      $type: stdout
      buffer:
        $type: memory
        flush_interval: 1s

equivalent to

<ROOT>
  <system>
    workers 2
  </system>
  <worker 1>
    <source>
      @type dummy
      dummy {"message":"hello"}
      tag "dummy1.t"
    </source>
    <filter dummy1.*>
      @type record_transformer
      <record>
        hostname TD-0283
        tag ${tag}
      </record>
    </filter>
    <match dummy1.*>
      @type stdout
      <buffer>
        @type "memory"
        flush_interval 1s
      </buffer>
    </match>
  </worker>
  <worker 1>
    <source>
      @type dummy
      @label @test
      tag "test1"
    </source>
    <source>
      @type dummy
      @label @test
      tag "test2"
    </source>
    <label @test>
      <match test2>
        @type stdout
        <buffer>
          @type "memory"
          flush_interval 1s
        </buffer>
      </match>
      <match test1>
        @type stdout
        <buffer>
          @type "memory"
          flush_interval 1s
        </buffer>
      </match>
    </label>
  </worker>
  <source>
    @type dummy
    tag "tag"
  </source>
  <match tag>
    @type stdout
    <buffer>
      @type "memory"
      flush_interval 1s
    </buffer>
  </match>
</ROOT>

Docs Changes:

needed

Release Note:

same as the title.

[ganmacs]

Since we have no resource to maintain two( or three) config format, I'm closing this for now.