Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Encode utf8 for configuration string values #1411

Merged
merged 3 commits into from
Jan 10, 2017
Merged

Encode utf8 for configuration string values #1411

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
49f4c30
ensure to provide utf-8 string values for configuration params define…
tagomoris Jan 10, 2017
85615b7
permission should be read as string, and re-parsed as octet integer
tagomoris Jan 10, 2017
d56eaa9
Overwrite parameter definitions by subclass sections correctly.
tagomoris Jan 10, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion lib/fluent/config/configure_proxy.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,7 @@ def merge(other) # self is base class, other is subclass
merged.configured_in_section = self.configured_in_section || other.configured_in_section

merged.argument = other.argument || self.argument
merged.params = other.params.merge(self.params)
merged.params = self.params.merge(other.params)
merged.defaults = self.defaults.merge(other.defaults)
merged.sections = {}
(self.sections.keys + other.sections.keys).uniq.each do |section_key|
Expand Down
4 changes: 2 additions & 2 deletions lib/fluent/config/types.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ def self.bool_value(str)
end
end

STRING_TYPE = Proc.new { |val, opts| val }
STRING_TYPE = Proc.new { |val, opts| val.to_s.encode(Encoding::UTF_8) }
ENUM_TYPE = Proc.new { |val, opts|
s = val.to_sym
list = opts[:list]
Expand All @@ -85,7 +85,7 @@ def self.bool_value(str)
value
else
case type
when :string then value.to_s
when :string then value.to_s.encode(Encoding::UTF_8)
when :integer then value.to_i
when :float then value.to_f
when :size then Config.size_value(value)
Expand Down
4 changes: 3 additions & 1 deletion lib/fluent/plugin/buf_file.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,9 @@ def configure(conf)
end
end

unless @dir_permission
if @dir_permission
@dir_permission = @dir_permission.to_i(8) if @dir_permission.is_a?(String)
else
@dir_permission = system_config.dir_permission || DIR_PERMISSION
end
end
Expand Down
4 changes: 2 additions & 2 deletions lib/fluent/plugin/buffer/file_chunk.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,12 +42,12 @@ class FileChunk < Chunk

def initialize(metadata, path, mode, perm: system_config.file_permission || FILE_PERMISSION, compress: :text)
super(metadata, compress: compress)
@permission = perm
@permission = perm.is_a?(String) ? perm.to_i(8) : perm
@bytesize = @size = @adding_bytes = @adding_size = 0
@meta = nil

case mode
when :create then create_new_chunk(path, perm)
when :create then create_new_chunk(path, @permission)
when :staged then load_existing_staged_chunk(path)
when :queued then load_existing_enqueued_chunk(path)
else
Expand Down
1 change: 1 addition & 0 deletions test/config/test_types.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,7 @@ class TestConfigTypes < ::Test::Unit::TestCase
assert_equal 'test', Config::STRING_TYPE.call('test', {})
assert_equal '1', Config::STRING_TYPE.call('1', {})
assert_equal ' ', Config::STRING_TYPE.call(' ', {})
assert_equal Encoding::UTF_8, Config::STRING_TYPE.call('test', {}).encoding
end

test 'enum' do
Expand Down
4 changes: 2 additions & 2 deletions test/plugin/test_buf_file.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,7 @@ def write_metadata(path, chunk_id, metadata, size, ctime, mtime)
FileUtils.rm_r bufdir if File.exist?(bufdir)
assert !File.exist?(bufdir)

plugin.configure(config_element('buffer', '', {'path' => bufpath, 'dir_permission' => 0700}))
plugin.configure(config_element('buffer', '', {'path' => bufpath, 'dir_permission' => '0700'}))
assert !File.exist?(bufdir)

plugin.start
Expand Down Expand Up @@ -215,7 +215,7 @@ def write_metadata(path, chunk_id, metadata, size, ctime, mtime)
FileUtils.rm_r bufdir if File.exist?(bufdir)
assert !File.exist?(bufdir)

plugin.configure(config_element('buffer', '', {'path' => bufpath, 'file_permission' => 0600}))
plugin.configure(config_element('buffer', '', {'path' => bufpath, 'file_permission' => '0600'}))
assert !File.exist?(bufdir)
plugin.start

Expand Down