Skip to content

HDDS-2035 Implement datanode level CLI to reveal pipeline relation. #348

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
ChenSammi merged 2 commits into from
Dec 18, 2019
Merged

HDDS-2035 Implement datanode level CLI to reveal pipeline relation. #348

ChenSammi merged 2 commits into from
Dec 18, 2019

Conversation

@timmylicheng
Copy link
Contributor

@timmylicheng timmylicheng commented Dec 12, 2019

What changes were proposed in this pull request?

#HDDS-2035 Implement datanode level CLI to reveal pipeline info

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-2035

(Please create an issue in ASF JIRA before opening a pull request,
and you need to set the title of the pull request which starts with
the corresponding JIRA issue number. (e.g. HDDS-XXXX. Fix a typo in YYY.)

Please replace this section with the link to the Apache JIRA)

How was this patch tested?

bash-4.2$ ozone scmcli datanode list
Datanode: 7a68a668-3dbe-4366-9fed-271f221c1db0 (172.20.0.2/ozone_datanode_1.ozone_default).
Related pipelines:
cd060313-b724-46cf-a83a-8764ba066ae2/ONE/RATIS/OPEN

(Please explain how this patch was tested. Ex: unit tests, manual tests)
(If this patch involves UI changes, please attach a screen-shot; otherwise, remove this)

adoroszlai
adoroszlai reviewed Dec 13, 2019
View reviewed changes
Copy link
Contributor

@adoroszlai adoroszlai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @timmylicheng for working on this.

@timmylicheng timmylicheng force-pushed the HDDS-2035 branch 3 times, most recently from 501b642 to 340f5e8 Compare December 17, 2019 03:44
@timmylicheng
Copy link
Contributor Author

timmylicheng commented Dec 17, 2019

testCloseContainerEventWithRatis is fixed in #291

@ChenSammi
Copy link
Contributor

ChenSammi commented Dec 17, 2019

Command line filter doesn't take effect.

bash-4.2$ ozone scmcli datanode list --id=aaaa

Datanode: a37d1f73-7b4c-459a-a119-cd33c0fd64da (172.18.0.4/ozone_datanode_1.ozone_default/1 pipelines).
Related pipelines:
a1065d21-77a5-4265-80db-d341ac7c3da0/ONE/RATIS/OPEN/Leader

adoroszlai
adoroszlai requested changes Dec 17, 2019
View reviewed changes
Copy link
Contributor

@adoroszlai adoroszlai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stream.filter creates a new stream, which needs to be assigned to the existing stream to let the final forEach see the result of filtering.

@timmylicheng
Copy link
Contributor Author

timmylicheng commented Dec 17, 2019

Stream.filter creates a new stream, which needs to be assigned to the existing stream to let the final forEach see the result of filtering.

@adoroszlai I have made the changes you requested.

@timmylicheng
Copy link
Contributor Author

timmylicheng commented Dec 17, 2019 •
edited
Loading

Command line filter doesn't take effect.

bash-4.2$ ozone scmcli datanode list --id=aaaa

Datanode: a37d1f73-7b4c-459a-a119-cd33c0fd64da (172.18.0.4/ozone_datanode_1.ozone_default/1 pipelines).
Related pipelines:
a1065d21-77a5-4265-80db-d341ac7c3da0/ONE/RATIS/OPEN/Leader

bash-4.2$ ozone scmcli datanode list
Datanode: 63b02976-3fee-48dc-9f21-b42d7b6e4a40 (172.31.0.3/ozone_datanode_1.ozone_default/1 pipelines).
Related pipelines:
c8455063-62cc-4bb2-a63c-50a9d59a3adf/ONE/RATIS/ALLOCATED/Follower

bash-4.2$ ozone scmcli datanode list --ip 1.1.1.1

@ChenSammi Just updated.

adoroszlai
adoroszlai approved these changes Dec 17, 2019
View reviewed changes
Copy link
Contributor

@adoroszlai adoroszlai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @timmylicheng for updating the patch. Tried it locally applied to master, works fine.

@ChenSammi
Copy link
Contributor

ChenSammi commented Dec 18, 2019

Thanks @timmylicheng for the contribution. Committed to branch.

@ChenSammi ChenSammi merged commit e7315e9 into apache:HDDS-1564 Dec 18, 2019
@timmylicheng timmylicheng deleted the HDDS-2035 branch December 19, 2019 02:25
timmylicheng added a commit to timmylicheng/hadoop-ozone that referenced this pull request Dec 23, 2019
@timmylicheng
HDDS-2035 Implement datanode level CLI to reveal pipeline relation. (a…
8c07782 
…pache#348)
timmylicheng added a commit to timmylicheng/hadoop-ozone that referenced this pull request Jan 7, 2020
@timmylicheng
HDDS-2035 Implement datanode level CLI to reveal pipeline relation. (a…
3796154 
…pache#348)
timmylicheng added a commit to timmylicheng/hadoop-ozone that referenced this pull request Feb 10, 2020
@timmylicheng
HDDS-2035 Implement datanode level CLI to reveal pipeline relation. (a…
81ed727 
…pache#348)
timmylicheng added a commit to timmylicheng/hadoop-ozone that referenced this pull request Feb 12, 2020
@timmylicheng
HDDS-2035 Implement datanode level CLI to reveal pipeline relation. (a…
e720e7a 
…pache#348)
anuengineer pushed a commit that referenced this pull request Feb 19, 2020
@timmylicheng @ChenSammi @xiaoyuyao
HDDS-1564 Multiraft support (#538)
4ba1932 
* HDDS-1577. Add default pipeline placement policy implementation. (#1366)



(cherry picked from commit b640a5f6d53830aee4b9c2a7d17bf57c987962cd)

* HDDS-1571. Create an interface for pipeline placement policy to support network topologies. (#1395)

(cherry picked from commit 753fc6703a39154ed6013e44dbae572391748906)

* HDDS-2089: Add createPipeline CLI. (#1418)

(cherry picked from commit 326b5acd4a63fe46821919322867f5daff30750c)

* HDDS-1569 Support creating multiple pipelines with same datanode. Contributed by Li Cheng. 

This closes #28

* HDDS-1572 Implement a Pipeline scrubber to clean up non-OPEN pipeline. (#237)

* Rebase Fix

* HDDS-2650 Fix createPipeline CLI. (#340)

* HDDS-2035 Implement datanode level CLI to reveal pipeline relation. (#348)

* Revert "HDDS-2650 Fix createPipeline CLI. (#340)"

This reverts commit 7c71710.

* HDDS-2650 Fix createPipeline CLI and make it message based. (#370)

* HDDS-1574 Average out pipeline allocation on datanodes and add metrcs/test (#291)

* Resolve rebase conflict.

* HDDS-2756. Handle pipeline creation failure in different way when it exceeds pipeline limit

Closes #401

* HDDS-2115 Add acceptance test for createPipeline CLI and datanode list CLI (#375)

* HDDS-2115 Add acceptance test for createPipeline CLI and datanode list CLI.

* HDDS-2772 Better management for pipeline creation limitation. (#410)

*  HDDS-2913 Update config names and CLI for multi-raft feature. (#462)

* HDDS-2924. Fix Pipeline#nodeIdsHash collision issue. (#478)

* HDDS-2923 Add fall-back protection for rack awareness in pipeline creation. (#516)

* HDDS-3007 Fix CI test failure for TestSCMNodeManager. (#550)

Co-authored-by: Sammi Chen <[email protected]>
Co-authored-by: Xiaoyu Yao <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@adoroszlai adoroszlai adoroszlai approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@timmylicheng @ChenSammi @adoroszlai