HBASE-24807 Backport HBASE-20417 to branch-1 #2197
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bharathv
approved these changes
Aug 4, 2020
| try (WALEntryStream entryStream = | ||
| new WALEntryStream(logQueue, fs, conf, lastReadPosition, metrics)) { | ||
| while (isReaderRunning()) { // loop here to keep reusing stream while we can | ||
| if (!source.isPeerEnabled()) { |
Contributor
There was a problem hiding this comment.
This is just a safeguard to prevent accumulation of batches right? No other implications of the patch that I can think of.
Contributor
Author
There was a problem hiding this comment.
Yeah and the accumulation of batches can lead to major problems, because it's been accounted on overall buffer usage by ReplicationSourceManager. If buffer usage reaches the quota limits, replication becomes stuck. And since we check the buffer usage at ReplicationSourceManager, that means a single buffer for all peers. If one peer is disabled, while other source peers were supposed to continue to get replicated edits, those source would also be stuck because of this, until an RS restart.
Contributor
There was a problem hiding this comment.
Makes sense. I've been following the jira updates on the issue that Josh created.
infraio
added
the
backport
Contributor
|
Let me re-trigger the build, it should work now. |
|
💔 -1 overall
This message was automatically generated. |
Contributor
Author
|
Checking on the UT failures. |
Contributor
Author
|
Test failures look unrelated, have those tests passing, locally. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.