Skip to content

HDDS-8324. DN data cache gets removed randomly asking for data from disk #4499

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
szetszwo merged 4 commits into from
Apr 3, 2023
Merged

HDDS-8324. DN data cache gets removed randomly asking for data from disk #4499

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
fdd32b4
HDDS-8324. DN data cache gets removed randomly asking for data from disk
sumitagrawl Mar 30, 2023
a9e428f
HDDS-8324. DN data cache gets removed randomly asking for data from disk
sumitagrawl Mar 31, 2023
7f64910
Merge remote-tracking branch 'origin/master' into HDDS-8324
adoroszlai Apr 1, 2023
6728d36
Extract method removeCacheDataUpTo
adoroszlai Apr 1, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 15 additions & 5 deletions ...rg/apache/hadoop/ozone/container/common/transport/server/ratis/ContainerStateMachine.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -851,7 +851,7 @@ private void removeStateMachineDataIfNeeded(long index) {
.getFollowerNextIndices()).min().getAsLong();
LOG.debug("Removing data corresponding to log index {} min index {} "
+ "from cache", index, minIndex);
stateMachineDataCache.removeIf(k -> k >= (Math.min(minIndex, index)));
removeCacheDataUpTo(Math.min(minIndex, index));
}
} catch (Exception e) {
throw new RuntimeException(e);
Expand All @@ -873,9 +873,7 @@ public CompletableFuture<Message> applyTransaction(TransactionContext trx) {
removeStateMachineDataIfNeeded(index);
// if waitOnBothFollower is false, remove the entry from the cache
// as soon as its applied and such entry exists in the cache.
if (!waitOnBothFollowers) {
stateMachineDataCache.removeIf(k -> k >= index);
}
removeStateMachineDataIfMajorityFollowSync(index);
DispatcherContext.Builder builder =
new DispatcherContext.Builder().setTerm(trx.getLogEntry().getTerm())
.setLogIndex(index);
Expand Down Expand Up @@ -982,6 +980,18 @@ public CompletableFuture<Message> applyTransaction(TransactionContext trx) {
}
}

private void removeStateMachineDataIfMajorityFollowSync(long index) {
if (!waitOnBothFollowers) {
// if majority follow in sync, remove all cache previous to current index
// including current index
removeCacheDataUpTo(index);
}
}

private void removeCacheDataUpTo(long index) {
stateMachineDataCache.removeIf(k -> k <= index);
}

private static <T> CompletableFuture<T> completeExceptionally(Exception e) {
final CompletableFuture<T> future = new CompletableFuture<>();
future.completeExceptionally(e);
Expand All @@ -996,7 +1006,7 @@ public void notifyNotLeader(Collection<TransactionContext> pendingEntries) {

@Override
public CompletableFuture<Void> truncate(long index) {
stateMachineDataCache.removeIf(k -> k >= index);
stateMachineDataCache.removeIf(k -> k > index);
return CompletableFuture.completedFuture(null);
}

Expand Down