HDDS-10514. Recon - Provide DN decommissioning detailed status and info inline with current CLI command output. #6376

devmadhuu · 2024-03-13T15:09:36Z

What changes were proposed in this pull request?

This PR provides the details information of all datanodes for which decommissioning is initiated.

This PR change has added two API endpoints to provide information of all datanodes for which decommissioning is initiated.

API: /api/v1/datanodes/decommission/info
Type: GET
This API provides information about all datanodes

{
        "datanodeDetails": {
            "uuid": "f04ed2ee-5d70-4e9b-88a8-d54dd59d2828",
            "hostname": "ozone-ha-datanode-1.ozone-ha_default",
            "lastHeartbeat": 0,
            "networkLocation": "/default-rack",
            "ipAddress": "192.168.208.12"
        },
        "metrics": {
            "decommissionStartTime": "13/03/2024 02:52:29 UTC",
            "numOfUnclosedPipelines": 2,
            "numOfUnderReplicatedContainers": 0.0,
            "numOfUnclosedContainers": 0.0
        },
        "containers": {
            "UnderReplicated": [],
            "UnClosed": [#6, #10, #17]
        }
    }

API: /api/v1/datanodes/decommission/info/<uuid>
Type: GET
This API provides information about a specific datanode for which decommissioning is initiated.

{
        "datanodeDetails": {
            "uuid": "f04ed2ee-5d70-4e9b-88a8-d54dd59d2828",
            "hostname": "ozone-ha-datanode-1.ozone-ha_default",
            "lastHeartbeat": 0,
            "networkLocation": "/default-rack",
            "ipAddress": "192.168.208.12"
        },
        "metrics": {
            "decommissionStartTime": "13/03/2024 02:52:29 UTC",
            "numOfUnclosedPipelines": 2,
            "numOfUnderReplicatedContainers": 0.0,
            "numOfUnclosedContainers": 0.0
        },
        "containers": {
            "UnderReplicated": [],
            "UnClosed": [#6, #10, #17]
        }
    }

Above information will be consumed by UI to display in Recon.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-10514

How was this patch tested?

This patch is tested with integration test as well as APIs endpoints Junit test cases.

…fo inline with current CLI command output.

devmadhuu · 2024-03-15T06:20:18Z

@dombizita @ArafatKhan2198 @sumitagrawl Kindly review.

This reverts commit c29ac0b.

devmadhuu · 2024-03-18T07:43:34Z

This PR is dependent on #6369

Tejaskriya · 2024-03-19T12:03:19Z

...dds/common/src/main/java/org/apache/hadoop/hdds/recon/CustomDatanodeDetailsDeserializer.java

+import java.util.UUID;
+
+/**
+ * This is custom jackson deserializer class for DetanodeDetails class.


Minor typo here for DatanodeDetails

Suggested change

* This is custom jackson deserializer class for DetanodeDetails class.

* This is custom jackson deserializer class for DatanodeDetails class.

Tejaskriya · 2024-03-19T12:03:43Z

...-hdds/common/src/main/java/org/apache/hadoop/hdds/recon/CustomDatanodeDetailsSerializer.java

+import java.io.IOException;
+
+/**
+ * This is custom jackson serializer class for DetanodeDetails class.


Same here too

Suggested change

* This is custom jackson serializer class for DetanodeDetails class.

* This is custom jackson serializer class for DatanodeDetails class.

sumitagrawl

@devmadhuu given few comments

hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/NodeEndpoint.java

hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/ReconUtils.java

smitajoshi12 · 2024-04-25T06:10:01Z

@devmadhuu

sumitagrawl

@devmadhuu Thanks for working over this, have few minor comments

hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/HddsUtils.java

hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/NodeEndpoint.java

hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/ReconServerConfigKeys.java

sumitagrawl

LGTM

…fo inline with current CLI command output. (apache#6376)

…concile-cli * HDDS-10239-container-reconciliation: (296 commits) HDDS-10897. Refactor OzoneQuota (apache#6714) HDDS-10422. Fix some warnings about exposing internal representation in hdds-common (apache#6351) HDDS-10899. Refactor Lease callbacks (apache#6715) HDDS-10890. Increase default value for hdds.container.ratis.log.appender.queue.num-elements (apache#6711) HDDS-10832. Client should switch to streaming based on OpenKeySession replication (apache#6683) HDDS-10435. Support S3 object tags for existing requests (apache#6607) HDDS-10883. Improve logging in Recon for finalising DN logic. (apache#6704) HDDS-8752. Enable TestOzoneRpcClientAbstract#testOverWriteKeyWithAndWithOutVersioning (apache#6702) HDDS-10875. XceiverRatisServer#getRaftPeersInPipeline should be called before XceiverRatisServer#removeGroup (apache#6696) HDDS-10514. Recon - Provide DN decommissioning detailed status and info inline with current CLI command output. (apache#6376) HDDS-10878. Bump zstd-jni to 1.5.6-3 (apache#6701) HDDS-10877. Bump Dropwizard metrics to 3.2.6 (apache#6699) HDDS-10876. Bump jackson to 2.16.2 (apache#6697) HDDS-6116. Remove flaky tag from TestSCMInstallSnapshot (apache#6695) HDDS-2643. TestOzoneDelegationTokenSecretManager#testRenewTokenFailureRenewalTime fails intermittently. HDDS-10699. Refactor ContainerBalancerTask and TestContainerBalancerTask (apache#6537) HDDS-10861. Ozone cli supports default ozone.om.service.id (apache#6680) HDDS-10859. Improve error messages when decommission and maintenance fail-early (apache#6678) HDDS-9031. Upgrade acceptance tests to Docker Compose v2 (apache#6667) HDDS-10559. Add a warning or a check to run repair tool as System user (apache#6574) ... Conflicts: hadoop-ozone/dist/src/main/smoketest/admincli/container.robot

deveshsingh added 2 commits March 13, 2024 20:28

HDDS-10514. Recon - Provide DN decommissioning detailed status and in…

523cc1d

…fo inline with current CLI command output.

HDDS-10514. Reverting this file change.

525ef35

adoroszlai added the recon label Mar 13, 2024

deveshsingh added 3 commits March 14, 2024 19:31

HDDS-10514. Added API for UUID argument.

f1df560

HDDS-10514. Fixed findbugs.

c229db3

HDDS-10514. Added test cases for endPoint APIs.

eb57e7a

devmadhuu marked this pull request as ready for review March 15, 2024 06:20

devmadhuu marked this pull request as draft March 15, 2024 16:23

deveshsingh added 4 commits March 16, 2024 09:26

HDDS-9537. Intermittent failure in TestPipelineManagerMXBean.

c29ac0b

Revert "HDDS-9537. Intermittent failure in TestPipelineManagerMXBean."

fe6ae72

This reverts commit c29ac0b.

HDDS-10514. Fixed JSON formatting error and test cases.

378ed7a

HDDS-10514. Fixed findbugs.

0f76ecc

devmadhuu marked this pull request as ready for review March 18, 2024 07:41

devmadhuu marked this pull request as draft March 18, 2024 10:27

HDDS-10514. Fixed test failed cases.

8e90c06

devmadhuu marked this pull request as ready for review March 18, 2024 11:56

Tejaskriya reviewed Mar 19, 2024

View reviewed changes

HDDS-10514. Fixed spelling mistakes in javadoc comments.

b1c22b8

sumitagrawl reviewed Apr 4, 2024

View reviewed changes

HDDS-10514. Fixed review comments.

0869a88

deveshsingh added 2 commits May 15, 2024 08:26

HDDS-10514. Changed the approach to get decommission status for DNs.

ed958bd

HDDS-10514. Added test cases.

0ab6ae8

sumitagrawl reviewed May 15, 2024

View reviewed changes

hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/HddsUtils.java Outdated Show resolved Hide resolved

hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/NodeEndpoint.java Show resolved Hide resolved

deveshsingh added 2 commits May 15, 2024 20:15

HDDS-10514. Handled review comments.

6513cc3

HDDS-10514. Handled review comments.

aa1b62b

sumitagrawl reviewed May 17, 2024

View reviewed changes

hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/ReconServerConfigKeys.java Outdated Show resolved Hide resolved

HDDS-10514. Removed unused constant.

b4bc438

sumitagrawl approved these changes May 17, 2024

View reviewed changes

sumitagrawl merged commit 6486be8 into apache:master May 19, 2024

jojochuang pushed a commit to jojochuang/ozone that referenced this pull request May 23, 2024

HDDS-10514. Recon - Provide DN decommissioning detailed status and in…

15a51fe

…fo inline with current CLI command output. (apache#6376)

smitajoshi12 mentioned this pull request May 24, 2024

HDDS-10517. Recon - Add a UI component for showing DN decommissioning detailed status and info #6724

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

HDDS-10514. Recon - Provide DN decommissioning detailed status and info inline with current CLI command output. #6376

HDDS-10514. Recon - Provide DN decommissioning detailed status and info inline with current CLI command output. #6376

Uh oh!

devmadhuu commented Mar 13, 2024 •

edited

Loading

Uh oh!

devmadhuu commented Mar 15, 2024

Uh oh!

devmadhuu commented Mar 18, 2024

Uh oh!

Tejaskriya Mar 19, 2024

Uh oh!

Tejaskriya Mar 19, 2024

Uh oh!

sumitagrawl left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

smitajoshi12 commented Apr 25, 2024

Uh oh!

sumitagrawl left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sumitagrawl left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

	* This is custom jackson deserializer class for DetanodeDetails class.
	* This is custom jackson deserializer class for DatanodeDetails class.

	* This is custom jackson serializer class for DetanodeDetails class.
	* This is custom jackson serializer class for DatanodeDetails class.

HDDS-10514. Recon - Provide DN decommissioning detailed status and info inline with current CLI command output. #6376

HDDS-10514. Recon - Provide DN decommissioning detailed status and info inline with current CLI command output. #6376

Uh oh!

Conversation

devmadhuu commented Mar 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes were proposed in this pull request?

What is the link to the Apache JIRA

How was this patch tested?

Uh oh!

devmadhuu commented Mar 15, 2024

Uh oh!

devmadhuu commented Mar 18, 2024

Uh oh!

Tejaskriya Mar 19, 2024

Choose a reason for hiding this comment

Uh oh!

Tejaskriya Mar 19, 2024

Choose a reason for hiding this comment

Uh oh!

sumitagrawl left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

smitajoshi12 commented Apr 25, 2024

Uh oh!

sumitagrawl left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sumitagrawl left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

devmadhuu commented Mar 13, 2024 •

edited

Loading