Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1.0: Fix wrong description of total retry time #395

Merged
merged 3 commits into from
Mar 31, 2022
+55 −6
Merged

1.0: Fix wrong description of total retry time #395

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
c8f16a3
Fix wrong description of total retry time
daipom Mar 25, 2022
a6aa7e9
Fix wrong number of retry times
daipom Mar 31, 2022
06de24a
Add description about last retry and add examples
daipom Mar 31, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion buffer/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ A chunk can fail to be written out to the destination for a number of reasons. T
By default, Fluentd increases the wait interval exponentially for each retry attempt. For example, assuming that the initial wait interval is set to 1 second and the exponential factor is 2, each attempt occurs at the following time points:

```text
1 2 4 8 16
0 1 3 7 15
x-x---x-------x---------------x-------------------------
│ │ │ │ └─ 4th retry (wait = 8s)
│ │ │ └───────────────── 3th retry (wait = 4s)
Expand Down
9 changes: 7 additions & 2 deletions configuration/buffer-section.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -494,7 +494,11 @@ Following are the flushing parameters for chunks to optimize performance \(laten
* Default: 72h
* The maximum time \(seconds\) to retry to flush again the failed chunks,

until the plugin discards the buffer chunks
until the plugin discards the buffer chunks.

If the next retry is going to exceed this time limit, the last retry

will be made at exactly this time limit.
* `retry_forever` \[bool\]
* Default: `false`
* If true, plugin will ignore `retry_timeout` and `retry_max_times`
Expand Down Expand Up @@ -546,7 +550,8 @@ With `exponential_backoff`, `retry_wait` interval will be calculated as below:
* c: constant factor, `@retry_wait`
* b: base factor, `@retry_exponential_backoff_base`
* k: number of retry times
* total retry time: `c + c * b^1 + (...) + c*b^k = c*b^(k+1) - 1`
* total retry time: `c + c*b^1 + (...) + c*b^(k-1) = c*(b^k - 1) / (b - 1)`
* = `2^k - 1` by default

If this article is incorrect or outdated, or omits critical information, please [let us know](https://github.com/fluent/fluentd-docs-gitbook/issues?state=open). [Fluentd](http://www.fluentd.org/) is an open-source project under [Cloud Native Computing Foundation \(CNCF\)](https://cncf.io/). All components are available under the Apache 2 License.

50 changes: 47 additions & 3 deletions output/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -181,6 +181,18 @@ If the bottom chunk write out fails, it will remain in the queue and Fluentd wil

Writing out the bottom chunk is considered to be a failure if `Output#write` or `Output#try_write` method throws an exception.

The retry timings of `retry_timeout: 100s`.

| N-th retry | Elapsed |
| :--- | :--- |
| 1th | 1s |
| 2th | 3s |
| 3th | 7s |
| 4th | 15s |
| 5th | 31s |
| 6th | 63s |
| 7th | 100s |

#### `retry_type`

Specifies how to wait for the next retry to flush buffer.
Expand All @@ -198,13 +210,13 @@ Default: `false`

#### `retry_timeout`

The maximum seconds to retry to flush while failing, until the plugin discards the buffer chunks.
The maximum seconds to retry to flush while failing, until the plugin discards the buffer chunks. If the next retry is going to exceed this time limit, the last retry will be made at exactly this time limit.

Default: `72` \(hours\)
Default: `72h` \(72 hours\)

#### `retry_max_times`

The maximum number of times to retry to flush while failing. If `retry_timeout` is the default, the number is 17 with exponential backoff.
The maximum number of times to retry to flush while failing. If `retry_timeout` is the default, the number is 18 with exponential backoff.

Default: `nil`

Expand Down Expand Up @@ -268,5 +280,37 @@ This example sends logs to Elasticsearch using a file buffer `/var/log/td-agent/

NOTE: `<secondary>` plugin receives the primary's buffer chunk directly. So, you need to check if your secondary plugin works with the primary setting.

The retry timings of `retry_timeout: 100s` with the secondary.

| N-th retry | Elapsed | Output plugin |
| :--- | :--- | :--- |
| 1th | 1s | primary |
| 2th | 3s | primary |
| 3th | 7s | primary |
| 4th | 15s | primary |
| 5th | 31s | primary |
| 6th | 63s | primary |
| 7th | 80s | secondary |
| 8th | 81s | secondary |
| 9th | 83s | secondary |
| 10th | 87s | secondary |
| 11th | 95s | secondary |
| 12th | 100s | secondary |

The retry timings of `retry_max_times: 10` with the secondary.

| N-th retry | Elapsed | Output plugin |
| :--- | :--- | :--- |
| 1th | 1s | primary |
| 2th | 3s | primary |
| 3th | 7s | primary |
| 4th | 15s | primary |
| 5th | 31s | primary |
| 6th | 63s | primary |
| 7th | 127s | primary |
| 8th | 255s | primary |
| 9th | 511s | primary |
| 10th | 818s | secondary |

If this article is incorrect or outdated, or omits critical information, please [let us know](https://github.com/fluent/fluentd-docs-gitbook/issues?state=open). [Fluentd](http://www.fluentd.org/) is an open-source project under [Cloud Native Computing Foundation \(CNCF\)](https://cncf.io/). All components are available under the Apache 2 License.