Bug: Polling only activates single thread, should eagerly create additional threads when jobs exist#1148
Merged
bensheldon merged 1 commit intomainfrom Nov 14, 2023
Merged
Bug: Polling only activates single thread, should eagerly create additional threads when jobs exist#1148bensheldon merged 1 commit intomainfrom
bensheldon merged 1 commit intomainfrom
Conversation
bensheldon
force-pushed
the
poll_fanout
branch
2 times, most recently
from
415f0db to
249e19a
Compare
bensheldon
force-pushed
the
poll_fanout
branch
from
249e19a to
a38976f
Compare
Contributor
|
Thank you for the heads up! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I think this is a bug. When only using polling (e.g. #810's
config.good_job.enable_listen_notify = false), only a single thread is ever active. This is because the Poller only ever activates a single thread, and then when a job is complete, it will only then activate the same thread again.This PR changes that behavior: now, when the poller triggers a Scheduler to activate a thread to query for a job, and a job is found, a 2nd thread will be activated before performing the first job. If there are significant jobs, this should fan out until all available threads are used.
I am classifying this as a bug, though it only affects performance/throughput. It's possible that this may cause your GoodJob processes to use more resources if you are relying on polling alone.
fyi, @mitchellhenke