Skip to content

HDFS-16642. [SBN read] Moving the selecting inputstream from journalnode in EditLogTailer out of FSNLock #4497

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ferhui merged 1 commit into from
Aug 4, 2022
Merged

HDFS-16642. [SBN read] Moving the selecting inputstream from journalnode in EditLogTailer out of FSNLock #4497

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 24 additions & 21 deletions ...ct/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/EditLogTailer.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -328,34 +328,37 @@ public Void run() throws Exception {

@VisibleForTesting
public long doTailEdits() throws IOException, InterruptedException {
Collection<EditLogInputStream> streams;
FSImage image = namesystem.getFSImage();

long lastTxnId = image.getLastAppliedTxId();
LOG.debug("lastTxnId: {}", lastTxnId);
long startTime = timer.monotonicNow();
try {
streams = editLog.selectInputStreams(lastTxnId + 1, 0,
null, inProgressOk, true);
} catch (IOException ioe) {
// This is acceptable. If we try to tail edits in the middle of an edits
// log roll, i.e. the last one has been finalized but the new inprogress
// edits file hasn't been started yet.
LOG.warn("Edits tailer failed to find any streams. Will try again " +
"later.", ioe);
return 0;
} finally {
NameNode.getNameNodeMetrics().addEditLogFetchTime(
timer.monotonicNow() - startTime);
}
// Write lock needs to be interruptible here because the
// transitionToActive RPC takes the write lock before calling
// tailer.stop() -- so if we're not interruptible, it will
// deadlock.
namesystem.writeLockInterruptibly();
try {
FSImage image = namesystem.getFSImage();

long lastTxnId = image.getLastAppliedTxId();

if (LOG.isDebugEnabled()) {
LOG.debug("lastTxnId: " + lastTxnId);
}
Collection<EditLogInputStream> streams;
long startTime = timer.monotonicNow();
try {
streams = editLog.selectInputStreams(lastTxnId + 1, 0,
null, inProgressOk, true);
} catch (IOException ioe) {
// This is acceptable. If we try to tail edits in the middle of an edits
// log roll, i.e. the last one has been finalized but the new inprogress
// edits file hasn't been started yet.
LOG.warn("Edits tailer failed to find any streams. Will try again " +
"later.", ioe);
long currentLastTxnId = image.getLastAppliedTxId();
if (lastTxnId != currentLastTxnId) {
LOG.warn("The currentLastTxnId({}) is different from preLastTxtId({})",
currentLastTxnId, lastTxnId);
return 0;
} finally {
NameNode.getNameNodeMetrics().addEditLogFetchTime(
timer.monotonicNow() - startTime);
}
if (LOG.isDebugEnabled()) {
LOG.debug("edit streams to load from: " + streams.size());
Expand Down