Skip to content

HDDS-2650 Fix createPipeline and make createPipeline CLI message based. #370

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
ChenSammi merged 1 commit into from
Dec 18, 2019
Merged

HDDS-2650 Fix createPipeline and make createPipeline CLI message based. #370

ChenSammi merged 1 commit into from
Dec 18, 2019

Conversation

@timmylicheng
Copy link
Contributor

@timmylicheng timmylicheng commented Dec 17, 2019 •
edited
Loading

What changes were proposed in this pull request?

#HDDS-2650 Make createPipeline CLI message based to fix after merge with master.

(Please fill in changes proposed in this fix)

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-2650

(Please create an issue in ASF JIRA before opening a pull request,
and you need to set the title of the pull request which starts with
the corresponding JIRA issue number. (e.g. HDDS-XXXX. Fix a typo in YYY.)

Please replace this section with the link to the Apache JIRA)

How was this patch tested?

bash-4.2$ ozone scmcli pipeline list
Pipeline[ Id: 2aa5b959-8e9d-4c72-a6ef-ec30f00f1dca, Nodes: 53ab812b-6ec4-468b-861c-64d2a1445deb{ip: 192.168.32.2, host: ozone_datanode_1.ozone_default, networkLocation: /default-rack, certSerialId: null}, Type:RATIS, Factor:ONE, State:ALLOCATED, leaderId:null, CreationTimestamp1576579695167]
bash-4.2$ ozone scmcli pipeline create
bash-4.2$ ozone scmcli pipeline list
Pipeline[ Id: 2aa5b959-8e9d-4c72-a6ef-ec30f00f1dca, Nodes: 53ab812b-6ec4-468b-861c-64d2a1445deb{ip: 192.168.32.2, host: ozone_datanode_1.ozone_default, networkLocation: /default-rack, certSerialId: null}, Type:RATIS, Factor:ONE, State:OPEN, leaderId:53ab812b-6ec4-468b-861c-64d2a1445deb, CreationTimestamp1576579705686]
Pipeline[ Id: c34c276f-f187-4048-9e20-81188c24fe79, Nodes: 53ab812b-6ec4-468b-861c-64d2a1445deb{ip: 192.168.32.2, host: ozone_datanode_1.ozone_default, networkLocation: /default-rack, certSerialId: null}, Type:STAND_ALONE, Factor:ONE, State:OPEN, leaderId:null, CreationTimestamp1576579705686]

(Please explain how this patch was tested. Ex: unit tests, manual tests)
(If this patch involves UI changes, please attach a screen-shot; otherwise, remove this)

@timmylicheng timmylicheng changed the title Make createPipeline CLI message based. HDDS-2650 Make createPipeline CLI message based. Dec 17, 2019
@ChenSammi
Copy link
Contributor

ChenSammi commented Dec 17, 2019 •
edited
Loading

Can you revert the previous HDDS-2650 commit and merge these two PR into one? Cannnot commit the same JR twice.

@timmylicheng
Copy link
Contributor Author

timmylicheng commented Dec 17, 2019

Can you revert the previous HDDS-2650 commit and merge these two PR into one? Cannnot commit the same JR twice.

I have included the revert commit and the new commit in this PR and I've updated the JIRA as well.

@timmylicheng
Copy link
Contributor Author

timmylicheng commented Dec 18, 2019

testCloseContainerEventWithRati is fixed in https://github.com/apache/hadoop-ozone/pull/291/files

@timmylicheng timmylicheng changed the title HDDS-2650 Make createPipeline CLI message based. HDDS-2650 Fix createPipeline and make createPipeline CLI message based. Dec 18, 2019
@ChenSammi
Copy link
Contributor

ChenSammi commented Dec 18, 2019

Can we refine the message when pipeline cannot be allocated for some reason?
bash-4.2$ ozone scmcli pipeline create -f THREE -t RATIS
Not enough healthy nodes to allocate pipeline. 3 datanodes required. Found 1
bash-4.2$ ozone scmcli pipeline create -f ONE -t RATIS
Cannot create pipeline of factor 1 using 0 nodes. Used 1 nodes. Healthy nodes 1

@timmylicheng
Copy link
Contributor Author

timmylicheng commented Dec 18, 2019

Can we refine the message when pipeline cannot be allocated for some reason?
bash-4.2$ ozone scmcli pipeline create -f THREE -t RATIS
Not enough healthy nodes to allocate pipeline. 3 datanodes required. Found 1
bash-4.2$ ozone scmcli pipeline create -f ONE -t RATIS
Cannot create pipeline of factor 1 using 0 nodes. Used 1 nodes. Healthy nodes 1

https://issues.apache.org/jira/browse/HDDS-2756 is tracking logging issues. Will address there.

@ChenSammi
Copy link
Contributor

ChenSammi commented Dec 18, 2019 •
edited
Loading

+1, will commit to the branch.

@ChenSammi ChenSammi merged commit 01eb8cd into apache:HDDS-1564 Dec 18, 2019
@timmylicheng timmylicheng deleted the HDDS-2650 branch December 19, 2019 02:25
timmylicheng added a commit to timmylicheng/hadoop-ozone that referenced this pull request Dec 23, 2019
@timmylicheng
HDDS-2650 Fix createPipeline CLI and make it message based. (apache#370)
f842691
timmylicheng added a commit to timmylicheng/hadoop-ozone that referenced this pull request Jan 7, 2020
@timmylicheng
HDDS-2650 Fix createPipeline CLI and make it message based. (apache#370)
fced9c1
timmylicheng added a commit to timmylicheng/hadoop-ozone that referenced this pull request Feb 10, 2020
@timmylicheng
HDDS-2650 Fix createPipeline CLI and make it message based. (apache#370)
6cdfdcb
timmylicheng added a commit to timmylicheng/hadoop-ozone that referenced this pull request Feb 12, 2020
@timmylicheng
HDDS-2650 Fix createPipeline CLI and make it message based. (apache#370)
1a27b25
anuengineer pushed a commit that referenced this pull request Feb 19, 2020
@timmylicheng @ChenSammi @xiaoyuyao
HDDS-1564 Multiraft support (#538)
4ba1932 
* HDDS-1577. Add default pipeline placement policy implementation. (#1366)



(cherry picked from commit b640a5f6d53830aee4b9c2a7d17bf57c987962cd)

* HDDS-1571. Create an interface for pipeline placement policy to support network topologies. (#1395)

(cherry picked from commit 753fc6703a39154ed6013e44dbae572391748906)

* HDDS-2089: Add createPipeline CLI. (#1418)

(cherry picked from commit 326b5acd4a63fe46821919322867f5daff30750c)

* HDDS-1569 Support creating multiple pipelines with same datanode. Contributed by Li Cheng. 

This closes #28

* HDDS-1572 Implement a Pipeline scrubber to clean up non-OPEN pipeline. (#237)

* Rebase Fix

* HDDS-2650 Fix createPipeline CLI. (#340)

* HDDS-2035 Implement datanode level CLI to reveal pipeline relation. (#348)

* Revert "HDDS-2650 Fix createPipeline CLI. (#340)"

This reverts commit 7c71710.

* HDDS-2650 Fix createPipeline CLI and make it message based. (#370)

* HDDS-1574 Average out pipeline allocation on datanodes and add metrcs/test (#291)

* Resolve rebase conflict.

* HDDS-2756. Handle pipeline creation failure in different way when it exceeds pipeline limit

Closes #401

* HDDS-2115 Add acceptance test for createPipeline CLI and datanode list CLI (#375)

* HDDS-2115 Add acceptance test for createPipeline CLI and datanode list CLI.

* HDDS-2772 Better management for pipeline creation limitation. (#410)

*  HDDS-2913 Update config names and CLI for multi-raft feature. (#462)

* HDDS-2924. Fix Pipeline#nodeIdsHash collision issue. (#478)

* HDDS-2923 Add fall-back protection for rack awareness in pipeline creation. (#516)

* HDDS-3007 Fix CI test failure for TestSCMNodeManager. (#550)

Co-authored-by: Sammi Chen <[email protected]>
Co-authored-by: Xiaoyu Yao <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@timmylicheng @ChenSammi