Skip to content

Commit

Permalink
Don't use timekey.object_id for Metadata instance comparison. Fix #2713
Browse files Browse the repository at this point in the history
Because this data will be differed unexpectedly on Windows.
This unexpected differing caused flood of buffer files
due to wrong metadata comparision.
`hash` method inside Metadata struct should be used on non-Windows environment.

This object_id value's unstability is monitored as follows:

```
{:timekey_object_id=>36247560}
{:timekey_object_id=>36247560}
{:timekey_object_id=>38199640}
{:timekey_object_id=>38199640}
{:timekey_object_id=>38199640}
{:timekey_object_id=>38240520}
{:timekey_object_id=>38240520}
{:timekey_object_id=>38240520}
{:timekey_object_id=>38277560}
{:timekey_object_id=>38277560}
{:timekey_object_id=>38277560}
{:timekey_object_id=>38314220}
{:timekey_object_id=>38314220}
{:timekey_object_id=>38314220}
{:timekey_object_id=>40539060}
{:timekey_object_id=>40539060}
{:timekey_object_id=>40539060}
{:timekey_object_id=>40598620}
{:timekey_object_id=>40598620}
{:timekey_object_id=>40598620}
{:timekey_object_id=>40764680}
{:timekey_object_id=>40764680}
{:timekey_object_id=>40764680}
{:timekey_object_id=>40613600}
{:timekey_object_id=>40613600}
{:timekey_object_id=>40613600}
{:timekey_object_id=>40741660}
{:timekey_object_id=>40741660}
{:timekey_object_id=>40741660}
{:timekey_object_id=>40895360}
{:timekey_object_id=>40895360}
{:timekey_object_id=>40895360}
{:timekey_object_id=>40926520}
<snip>
```

Signed-off-by: Hiroshi Hatake <[email protected]>
  • Loading branch information
cosmo0920 committed Jan 16, 2020
1 parent a353793 commit 941012a
Showing 1 changed file with 4 additions and 1 deletion.
5 changes: 4 additions & 1 deletion lib/fluent/plugin/buffer.rb
Original file line number Diff line number Diff line change
Expand Up @@ -138,9 +138,12 @@ def <=>(o)
# Actually this overhead is very small but this class is generated *per chunk* (and used in hash object).
# This means that this class is one of the most called object in Fluentd.
# See https://github.com/fluent/fluentd/pull/2560
# But, this optimization has a side effect on Windows due to differing object_id.
# This difference causes flood of buffer files.
# So, this optimization should be enabled on non-Windows platform.
def hash
timekey.object_id
end
end unless Fluent.windows?
end

# for tests
Expand Down

0 comments on commit 941012a

Please sign in to comment.