ignore_if_prev_successes not work

[hpv93]

Describe the bug

Hi all,
I want use output copy with ignore_if_prev_successes but it not work

To Reproduce

• I config 2 plugin output : forward and file
• Then I stop service of 2 agg node to set output forward fail

Expected behavior

• when plugin forward fail, i can't see log file is created by plugin 2

Your Environment

- Fluentd version: 1.13.3
- TD Agent version: 4.2.0
- Operating system: Ubuntu 20.04LTS
- Kernel version: 5.8.0-1038-gcp

Your Configuration

<source>
  @type tail
  path /var/log/nginx/test.demo.com.access.log
  pos_file /var/log/td-agent/nginx-access.log.pos
  refresh_interval 2
  tag apache.access
  <parse>
    @type json
  </parse>
</source>

<filter **.**>
  @type prometheus
  <metric>
    name fluentd_input_status_num_records_total
    type counter
    desc The total number of incoming records
    <labels>
      tag ${tag}
      hostname ${hostname}
    </labels>
  </metric>
</filter>

<match apache.access>
  @type copy
  <store ignore_error>
    @type forward
    send_timeout 10s
    recover_wait 3s
    hard_timeout 10s
    <buffer tag>
      @type file
      path /var/log/td-agent/buffer/
      flush_mode interval
      flush_interval 2s
      retry_max_times 3
    </buffer>

    <server>
      name agg-01
      host agg-01
      port 24224
      weight 50
    </server>
    <server>
      name agg-02
      host agg-02
      port 24224
      weight 50
    </server>
  </store>

  <store ignore_if_prev_success ignore_error>
    @type file
    path /var/log/td-agent/backup-store-%Y%m%d
    append true
    <buffer>
      @type memory
      flush_mode interval
      flush_interval 2s
    </buffer>
  </store>

  <store>
    @type prometheus
    <metric>
      name fluentd_output_status_num_records_total
      type counter
      desc The total number of outgoing records
      <labels>
        tag ${tag}
        hostname ${hostname}
      </labels>
    </metric>
  </store>
</match>

Your Error Log

2021-09-16 00:17:54 +0700 [warn]: #0 failed to flush the buffer. retry_time=0 next_retry_seconds=2021-09-16 00:17:55 +0700 chunk="5cc0be0d0313ee3826b88189fc91a53f" error_class=Fluent::Plugin::ForwardOutput::NoNodesAvailable error="no nodes are available"
  2021-09-16 00:17:54 +0700 [warn]: #0 suppressed same stacktrace
2021-09-16 00:17:55 +0700 [warn]: #0 failed to flush the buffer. retry_time=1 next_retry_seconds=2021-09-16 00:17:56 +0700 chunk="5cc0be0d0313ee3826b88189fc91a53f" error_class=Fluent::Plugin::ForwardOutput::NoNodesAvailable error="no nodes are available"
  2021-09-16 00:17:55 +0700 [warn]: #0 suppressed same stacktrace
2021-09-16 00:17:56 +0700 [warn]: #0 failed to flush the buffer. retry_time=2 next_retry_seconds=2021-09-16 00:17:58 +0700 chunk="5cc0be0d0313ee3826b88189fc91a53f" error_class=Fluent::Plugin::ForwardOutput::NoNodesAvailable error="no nodes are available"
  2021-09-16 00:17:56 +0700 [warn]: #0 suppressed same stacktrace
2021-09-16 00:17:58 +0700 [warn]: #0 failed to flush the buffer. retry_time=3 next_retry_seconds=2021-09-16 00:18:02 +0700 chunk="5cc0be0d0313ee3826b88189fc91a53f" error_class=Fluent::Plugin::ForwardOutput::NoNodesAvailable error="no nodes are available"
  2021-09-16 00:17:58 +0700 [warn]: #0 suppressed same stacktrace
2021-09-16 00:18:02 +0700 [error]: #0 failed to flush the buffer, and hit limit for retries. dropping all chunks in the buffer queue. retry_times=3 records=1 error_class=Fluent::Plugin::ForwardOutput::NoNodesAvailable error="no nodes are available"
  2021-09-16 00:18:02 +0700 [error]: #0 suppressed same stacktrace
^C

Additional context

No response

[ashie]

refs:
#3190
#3287

[ashie]

Buffered plugins never raise errors even if they reach retry_limit, so that ignore_if_prev_success doesn't work for this case. This feature isn't suitable for such case.

For your usage, secondary output is the suitable one to rescue failed messages.

[hpv93]

Ok, thank u. I will tried again with case write into buffer fail.