Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle both of stat and timer watchers are enabled and steady growing file case #3364

Merged
merged 7 commits into from
May 14, 2021
18 changes: 15 additions & 3 deletions lib/fluent/plugin/in_tail.rb
Original file line number Diff line number Diff line change
Expand Up @@ -908,6 +908,8 @@ def initialize(watcher, path:, read_lines_limit:, read_bytes_limit_per_second:,
@io = nil
@notify_mutex = Mutex.new
@log = log
@last_read_time = 0
@number_bytes_read = 0

@log.info "following tail of #{@path}"
end
Expand Down Expand Up @@ -948,18 +950,27 @@ def limit_bytes_per_second_reached?(start_reading, number_bytes_read)
false
end

def refresh_bytes_read_counter
if Fluent::Clock.now - @last_read_time > 1
@number_bytes_read = 0
end

yield
end

def handle_notify
with_io do |io|
cosmo0920 marked this conversation as resolved.
Show resolved Hide resolved
begin
number_bytes_read = 0
start_reading = Fluent::Clock.now
read_more = false

if !io.nil? && @lines.empty?
begin
while true
data = io.readpartial(BYTES_TO_READ, @iobuf)
number_bytes_read += data.bytesize
refresh_bytes_read_counter do
@number_bytes_read += data.bytesize
end
@fifo << data
@fifo.read_lines(@lines)

Expand All @@ -969,11 +980,12 @@ def handle_notify
read_more = true
break
end
if limit_bytes_per_second_reached?(start_reading, number_bytes_read)
if limit_bytes_per_second_reached?(start_reading, @number_bytes_read)
# Just get out from tailing loop.
read_more = false
break
end
@last_read_time = Fluent::Clock.now
ashie marked this conversation as resolved.
Show resolved Hide resolved
end
rescue EOFError
end
Expand Down