-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix Input and Output deadlock when buffer is full during startup #1502
Conversation
Output threads need calling `after_start` to run but `after_start` is sometimes not called when Input#start is blocked. This happens when in_tail with `read_from_head true` and output with `overflow_action block`. This fixes the problem by checking `after_start` immediately after start.
@tagomoris Do you see any concern for this patch? |
@repeatedly this change breaks many tests (makes tests unstable at least). read this commit comment: bd5841e IMO it's wrong to emit events in |
Which point? input or filter
Yes but current plugin API, it is hard, right? |
About the order of But it depends the order of About the threading, there are some options to do it - running long-running process in
The first option is more general solution. The second option is to call |
It seems good. I will implement this feature for in_tail like case in other patch. |
Update comment of |
lib/fluent/root_agent.rb
Outdated
# buffer related output threads should be run before `Input#start`. | ||
# This is why after_start should be called immediately after start call. | ||
# This depends on `desc: true` because calling plugin order of `desc: true` is | ||
# Output, Filter, Label, Output with Router and Input. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"Output with Router, then Input" looks better to emphasis the order.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added a trivial comment. LGTM.
Ah, please open an issue to add the |
cb8a2ce
to
ecf0a6a
Compare
Output threads need calling
after_start
to run butafter_start
is sometimes not called when Input#start is blocked.This happens when in_tail with
read_from_head true
andoutput with
overflow_action block
.This fixes the problem by checking
after_start
immediately after start.This may be related with #1485