Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
ed929c8
HDDS-5366. [Ozone-Streaming] Implement stream method to ContainerSta…
captainzmc Jun 23, 2021
cfd4444
update ozone ratis version to 2.1.0-d65ca26-SNAPSHOT
captainzmc Jun 8, 2021
c30661e
use ratis streaming
captainzmc Jun 29, 2021
956a11b
clean getDataStreamAddress code
kaijchen Jul 1, 2021
0bbe5ab
temporary workaround for NullPointerException
kaijchen Jul 1, 2021
d3fc826
fix default datastream port
kaijchen Jul 2, 2021
ceaa368
revert executePutBlock to synchronous
kaijchen Jul 2, 2021
f42862f
fix: wait stream close before sending PutBlock
kaijchen Jul 2, 2021
4790cdf
Revert "fix default datastream port"
kaijchen Jul 2, 2021
2e897ae
Revert "temporary workaround for NullPointerException"
kaijchen Jul 2, 2021
36ff249
fix dn start failed
captainzmc Jul 5, 2021
a5dce66
revert position of blockData build
kaijchen Jul 2, 2021
8c33358
fix port collision: default datastream port = 9855
kaijchen Jul 6, 2021
30a1810
remove unreachable code
kaijchen Jul 8, 2021
6c92326
proper way to get datastream port
kaijchen Jul 9, 2021
491aebc
add getDataStreamApi() in XceiverClientSpi interface
kaijchen Jul 9, 2021
0ad6059
revert delete newline
kaijchen Jul 9, 2021
51cc1ab
clean up code
kaijchen Jul 9, 2021
531ce94
revert adding extra try-catch
kaijchen Jul 9, 2021
09a65e7
clean up code
kaijchen Jul 9, 2021
66f3d20
better newRaftClient API
kaijchen Jul 14, 2021
21e2db5
link missing Override
captainzmc Jul 14, 2021
b949cd1
add Stream client framework
captainzmc Jul 14, 2021
bab4552
ozone sh put key support streaming.
captainzmc Jul 15, 2021
e199582
workaround for FORWARD message parsing error
kaijchen Jul 15, 2021
23f91c5
close stream before PutBlock
kaijchen Jul 20, 2021
58b1cb2
Merge branch 'fix-close-stream' into ozone-streaming
kaijchen Jul 21, 2021
05d109f
PutKeyHandler: print newline in streaming message
kaijchen Jul 21, 2021
04aff74
close stream before PutBlock
kaijchen Jul 21, 2021
d7cf401
BlockDataStreamOutput: close datastream asyncly
kaijchen Jul 21, 2021
3150307
fix stream closing properly
kaijchen Jul 22, 2021
d37a703
revert changes to BlockOutputStream
kaijchen Jul 22, 2021
0daacb9
abstract needSync
kaijchen Jul 28, 2021
df2031d
Merge branch 'ozone-streaming' into streaming-client
kaijchen Jul 29, 2021
f82435c
PutKeyHandler: prepare ByteBuf for streaming write
kaijchen Aug 3, 2021
3cb8c11
PutKeyHandler: split File into ByteBuf segments
kaijchen Aug 3, 2021
b7d990d
OzoneDataStreamOutput: support write ByteBuf (preferred)
kaijchen Aug 3, 2021
6c90901
DataStreamOutput: new interface
kaijchen Aug 3, 2021
0d4e381
createDataStreamOutput: drop support for crypto/cipher OutputStream
kaijchen Aug 3, 2021
a8e2643
KeyDataStreamOutput: implements DataStreamOutput
kaijchen Aug 3, 2021
56c721d
KeyDataStreamOutput: prioritize ByteBuf over []byte
kaijchen Aug 3, 2021
1957398
BlockDataStreamOutputEntry: implements DataStreamOutput
kaijchen Aug 3, 2021
12cb538
DataStreamOutput: default method write(ByteBuf, off, len)
kaijchen Aug 4, 2021
d2bf46d
BlockDataStreamOutput: implements DataStreamOutput
kaijchen Aug 4, 2021
8c12959
BlockDataStreamOutput: prioritize ByteBuf over []byte
kaijchen Aug 4, 2021
77c5613
PutKeyHandler: choose API (async/streaming) by comparing file length …
kaijchen Aug 4, 2021
375e9c8
BlockDataStreamOutput: annotate TODO
kaijchen Aug 4, 2021
e6a16e1
hdds-client: add dependency io.netty.buffer
kaijchen Aug 4, 2021
b7e5c3d
fix style check
kaijchen Aug 4, 2021
aa6a3e7
unify write(int) methods
kaijchen Aug 4, 2021
bb9d5b7
BlockDataStreamOutput: abstract setupStream()
kaijchen Aug 5, 2021
54efe91
BlockDataStreamOutput: disable disk sync
kaijchen Aug 5, 2021
f98b3ee
BlockDataStreamOutput: fix initial offset in write
kaijchen Aug 6, 2021
d91de0d
rename interface DataStreamOutput to ByteBufferStreamOutput
kaijchen Aug 6, 2021
787b630
DataStreamOutputs no longer extends OutputStream
kaijchen Aug 6, 2021
d9d4cd7
fix style check
kaijchen Aug 7, 2021
c197419
Merge branch 'HDDS-4454' into bytebuf-streaming-client
kaijchen Aug 10, 2021
dbacc90
update comments and javadoc
kaijchen Aug 10, 2021
c9d91ab
add ut for BlockDataStreamOutPut
captainzmc Aug 10, 2021
3e22953
trigger new CI
captainzmc Aug 10, 2021
82035b2
trigger new CI
captainzmc Aug 10, 2021
ee7686e
trigger new CI
captainzmc Aug 10, 2021
efe65ab
ByteBufferStreamOutput -> ByteBufStreamOutput
kaijchen Aug 11, 2021
f84936e
KeyDataStreamOutput: fix offset in write()
kaijchen Aug 11, 2021
7115b48
don't add deprecated method
kaijchen Aug 11, 2021
5f5cdac
TestHelper: stop using deprecated methods
kaijchen Aug 11, 2021
86a97a7
trigger new CI
captainzmc Aug 11, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions hadoop-hdds/client/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,10 @@ https://maven.apache.org/xsd/maven-4.0.0.xsd">
<version>${spotbugs.version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-buffer</artifactId>
</dependency>
</dependencies>

<build>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@
import com.google.common.annotations.VisibleForTesting;
import com.google.common.base.Preconditions;
import org.apache.ratis.client.RaftClient;
import org.apache.ratis.client.api.DataStreamApi;
import org.apache.ratis.grpc.GrpcTlsConfig;
import org.apache.ratis.proto.RaftProtos;
import org.apache.ratis.protocol.exceptions.GroupMismatchException;
Expand Down Expand Up @@ -359,4 +360,8 @@ public XceiverClientReply sendCommandAsync(
throw new UnsupportedOperationException(
"Operation Not supported for ratis client");
}

public DataStreamApi getDataStreamApi() {
return this.getClient().getDataStreamApi();
}
}
Loading