Skip to content

Commit

Permalink
Delete top level using bracket style properly
Browse files Browse the repository at this point in the history
In previous version, following configuration does not work properly:

```
<source>
  @type dummy
  dummy [
    {"foo.bar": "test1234", "message": "Hello"}
  ]
  tag dummy
</source>

<filter dummy>
  @type record_transformer
  remove_keys "$['foo.bar']"
</filter>

<match dummy>
  @type stdout
</match>
```

This shows like following:

```
2018-11-27 15:19:18 +0900 [info]: #0 fluentd worker is now running worker=0
2018-11-27 15:19:19.008586045 +0900 dummy: {"foo.bar":"test1234","message":"Hello"}
2018-11-27 15:19:20.009721132 +0900 dummy: {"foo.bar":"test1234","message":"Hello"}
2018-11-27 15:19:21.010784035 +0900 dummy: {"foo.bar":"test1234","message":"Hello"}
```

In this version, it works well.

See also #2109

Signed-off-by: Kenji Okimoto <[email protected]>
  • Loading branch information
okkez committed Nov 27, 2018
1 parent cf85ba6 commit f70e743
Show file tree
Hide file tree
Showing 2 changed files with 21 additions and 2 deletions.
10 changes: 8 additions & 2 deletions lib/fluent/plugin_helper/record_accessor.rb
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,14 @@ def initialize(param)
if @keys.is_a?(Array)
@last_key = @keys.last
@dig_keys = @keys[0..-2]
mcall = method(:call_dig)
mdelete = method(:delete_nest)
if @dig_keys.empty?
@keys = @keys.first
mcall = method(:call_index)
mdelete = method(:delete_top)
else
mcall = method(:call_dig)
mdelete = method(:delete_nest)
end
else
# Call [] for single key to reduce dig overhead
mcall = method(:call_index)
Expand Down
13 changes: 13 additions & 0 deletions test/plugin_helper/test_record_accessor.rb
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,12 @@ class Dummy < Fluent::Plugin::TestBase
assert_equal r[param], accessor.call(r)
end

test "access single dot key using bracket style" do
r = {'key1' => 'v1', 'ke y2' => 'v2', 'this.is.key3' => 'v3'}
accessor = @d.record_accessor_create('$["this.is.key3"]')
assert_equal 'v3', accessor.call(r)
end

test "nested bracket keys with dot" do
r = {'key1' => {'this.is.key3' => 'value'}}
accessor = @d.record_accessor_create("$['key1']['this.is.key3']")
Expand Down Expand Up @@ -164,6 +170,13 @@ class Dummy < Fluent::Plugin::TestBase
assert_not_include(r, param)
end

test "delete top key using bracket style" do
r = {'key1' => 'v1', 'ke y2' => 'v2', 'this.is.key3' => 'v3'}
accessor = @d.record_accessor_create('$["this.is.key3"]')
accessor.delete(r)
assert_not_include(r, 'this.is.key3')
end

data('bracket' => "$['key1'][0]['ke y2']",
'bracket w/ double quotes' => '$["key1"][0]["ke y2"]')
test "delete nested keys ['key1', 0, 'ke y2']" do |param|
Expand Down

0 comments on commit f70e743

Please sign in to comment.