Skip to content

HDDS-8220. [Ozone-Streaming] Trigger volume check on IOException in StreamDataChannelBase #4428

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
adoroszlai merged 4 commits into from
Mar 22, 2023
Merged

HDDS-8220. [Ozone-Streaming] Trigger volume check on IOException in StreamDataChannelBase #4428

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
b2363be
HDDS-8220. [Ozone-Streaming] KeyValueStreamDataChannel add write data…
guohao-rosicky Mar 20, 2023
8a7b5e9
trigger new CI
guohao-rosicky Mar 20, 2023
b043546
rename method
guohao-rosicky Mar 21, 2023
a9824aa
unused code
guohao-rosicky Mar 21, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 19 additions & 3 deletions .../main/java/org/apache/hadoop/ozone/container/keyvalue/impl/KeyValueStreamDataChannel.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,8 @@
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.concurrent.atomic.AtomicReference;

import static org.apache.hadoop.ozone.container.common.utils.StorageVolumeUtil.onFailure;

/**
* This class is used to get the DataChannel for streaming.
*/
Expand Down Expand Up @@ -158,7 +160,12 @@ ContainerProtos.Type getType() {
public int write(ReferenceCountedObject<ByteBuffer> referenceCounted)
throws IOException {
assertOpen();
return writeBuffers(referenceCounted, buffers, super::writeFileChannel);
try {
return writeBuffers(referenceCounted, buffers, super::writeFileChannel);
} catch (IOException e) {
volumeOnFailure();
throw e;
}
}

static int writeBuffers(ReferenceCountedObject<ByteBuffer> src,
Expand Down Expand Up @@ -198,8 +205,13 @@ void assertOpen() throws IOException {
@Override
public void close() throws IOException {
if (closed.compareAndSet(false, true)) {
putBlockRequest.set(closeBuffers(buffers, super::writeFileChannel));
super.close();
try {
putBlockRequest.set(closeBuffers(buffers, super::writeFileChannel));
super.close();

@hemantk-12 hemantk-12 Mar 20, 2023

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should super.close() be out of the try-catch?

} catch (IOException e) {
volumeOnFailure();
throw e;
}
}
}

Expand Down Expand Up @@ -274,4 +286,8 @@ private static ContainerCommandRequestProto readPutBlockRequest(ByteBuffer b)
}
return request;
}

private void volumeOnFailure() {

@hemantk-12 hemantk-12 Mar 20, 2023

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I don't think it is ideal to create function for one liner.

onFailure(getContainerData().getVolume());

@szetszwo szetszwo Mar 21, 2023

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@guohao-rosicky , thanks for working on this!

Let's move this method up and rename it to checkVolume.

+++ b/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/impl/StreamDataChannelBase.java
@@ -22,6 +22,7 @@
 import org.apache.hadoop.hdds.scm.container.common.helpers.StorageContainerException;
 import org.apache.hadoop.ozone.container.common.helpers.ContainerMetrics;
 import org.apache.hadoop.ozone.container.common.impl.ContainerData;
+import org.apache.hadoop.ozone.container.common.utils.StorageVolumeUtil;
 import org.apache.ratis.statemachine.StateMachine;
 
 import java.io.File;
@@ -64,6 +65,10 @@ private FileChannel getChannel() {
     return randomAccessFile.getChannel();
   }
 
+  void checkVolume() {
+    StorageVolumeUtil.onFailure(containerData.getVolume());
+  }
+
   @Override
   public final void force(boolean metadata) throws IOException {
     getChannel().force(metadata);

}
}
4 changes: 4 additions & 0 deletions .../src/main/java/org/apache/hadoop/ozone/container/keyvalue/impl/StreamDataChannelBase.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,10 @@ private FileChannel getChannel() {
return randomAccessFile.getChannel();
}

public ContainerData getContainerData() {
return containerData;
}

@Override
public final void force(boolean metadata) throws IOException {
getChannel().force(metadata);
Expand Down