Skip to content

[HBASE-25536] Remove 0 length wal file from logQueue if it belongs to… #2908

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
virajjasani merged 2 commits into from
Jan 29, 2021
Merged

[HBASE-25536] Remove 0 length wal file from logQueue if it belongs to… #2908

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
524f7b0
[HBASE-25536] Remove 0 length wal file from logQueue if it belongs to…
Jan 28, 2021
07fb533
[HBASE-25536] Adding test case
Jan 28, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ...ain/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceWALReader.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -248,7 +248,7 @@ private void handleEmptyWALEntryBatch() throws InterruptedException {
// enabled, then dump the log
private void handleEofException(IOException e) {
if ((e instanceof EOFException || e.getCause() instanceof EOFException) &&
logQueue.size() > 1 && this.eofAutoRecovery) {
(source.isRecovered() || logQueue.size() > 1) && this.eofAutoRecovery) {
Copy link
Contributor

@bharathv bharathv Jan 28, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice find, mind adding a quick comment since this is a subtle behavior?

Copy link
Contributor

@bharathv bharathv Jan 28, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On a related note, should we add a source and global metric counter to track the number of 0 size files dequeued? Seems more common than we thought, easy to monitor may be..

Copy link
Contributor Author

@shahrs87 shahrs87 Jan 28, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On a related note, should we add a source and global metric counter to track the number of 0 size files dequeued? Seems more common than we thought, easy to monitor may be..

@bharathv Good idea. Could you please create a separate jira to track this. Thank you !

try {
if (fs.getFileStatus(logQueue.peek()).getLen() == 0) {
LOG.warn("Forcing removal of 0 length log in queue: " + logQueue.peek());
Expand Down