HDDS-1868. Ozone pipelines should be marked as ready only after the leader election is complete. #23
Conversation
mukul1987
left a comment
mukul1987
left a comment
There was a problem hiding this comment.
The patch looks good to me.
...java/org/apache/hadoop/ozone/container/common/transport/server/ratis/XceiverServerRatis.java
Outdated
Show resolved
Hide resolved
...java/org/apache/hadoop/ozone/container/common/transport/server/ratis/XceiverServerRatis.java
Outdated
Show resolved
Hide resolved
nandakumar131
left a comment
nandakumar131
left a comment
There was a problem hiding this comment.
Overall the change looks ok.
...-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/PipelineStateManager.java
Outdated
Show resolved
Hide resolved
...e/src/main/java/org/apache/hadoop/ozone/container/common/report/PipelineReportPublisher.java
Outdated
Show resolved
Hide resolved
...java/org/apache/hadoop/ozone/container/common/transport/server/ratis/XceiverServerRatis.java
Outdated
Show resolved
Hide resolved
|
Thank you @mukul1987 and @nandakumar131 for the reviews. |
|
Thanks for updating the patch @swagle. There are some checkstyle issues in the latest patch and there are some test failures regarding pipeline esp along with the safe mode pipeline rules. Can you please take a look at the failures. |
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/MiniOzoneClusterImpl.java
Outdated
Show resolved
Hide resolved
...java/org/apache/hadoop/ozone/container/common/transport/server/ratis/XceiverServerRatis.java
Outdated
Show resolved
Hide resolved
...hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/PipelineReportHandler.java
Outdated
Show resolved
Hide resolved
...op-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/SCMPipelineManager.java
Outdated
Show resolved
Hide resolved
|
/retest |
|
/retest |
1 similar comment
|
/retest |
There was a problem hiding this comment.
There is no real change here, can we revert it?
There was a problem hiding this comment.
The leaderID was marked as optional.
There was a problem hiding this comment.
nitpick: We can make it strongly typed by making the type as DatanodeDetails
There was a problem hiding this comment.
But if I do that, in order to construct Pipeline from proto, I have to lookup DatanodeDetails based on UUID, which is a map in NodeManager. Not sure why making this into DatanodeDetails is better?
There was a problem hiding this comment.
It is just that it will be explicit what this UUID represent.
...hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/PipelineReportHandler.java
Outdated
Show resolved
Hide resolved
...op-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/SCMPipelineManager.java
Outdated
Show resolved
Hide resolved
|
/retest |
|
We can commit this after the Jenkins run. |
|
Test failure seems unrelated, merging this. |
5 participants
Ozone pipeline on create and restart, start in allocated state. They are moved into open state after all the pipeline have reported to it. However, this potentially can lead into an issue where the pipeline is still not ready to accept any incoming IO operations.
The pipelines should be marked as ready only after the leader election is complete and leader is ready to accept incoming IO.