HDDS-8127. Exclude deleted containers from Recon container count #4440
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
Author
|
@devmadhuu @sadanand48 @dombizita can you please take a look |
sadanand48
reviewed
Mar 21, 2023
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/ClusterStateEndpoint.java
Outdated
Show resolved
Hide resolved
devmadhuu
reviewed
Mar 21, 2023
...-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/types/ClusterStateResponse.java
Outdated
Show resolved
Hide resolved
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/ClusterStateEndpoint.java
Outdated
Show resolved
Hide resolved
adoroszlai
changed the title
Contributor
Author
|
@sadanand48 @ChenSammi @smengcl can take a look please! |
devmadhuu
reviewed
Mar 29, 2023
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/ClusterStateEndpoint.java
Outdated
Show resolved
Hide resolved
sumitagrawl
requested changes
Mar 29, 2023
Contributor
sumitagrawl
left a comment
sumitagrawl
left a comment
There was a problem hiding this comment.
@ArafatKhan2198 we can combine 3 test class to single
...-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/types/ClusterStateResponse.java
Outdated
Show resolved
Hide resolved
ArafatKhan2198
force-pushed
the
HDDS-8127
branch
from
d6d9115 to
d1b876c
Compare
sumitagrawl
approved these changes
Mar 31, 2023
Contributor
sumitagrawl
left a comment
sumitagrawl
left a comment
There was a problem hiding this comment.
@ArafatKhan2198 Thanks working on this, LGTM +1
Contributor
|
@ArafatKhan2198 @ChenSammi @sumitagrawl it looks like a flaky test was introduced here: |
errose28
added a commit
to errose28/ozone
that referenced
this pull request
Apr 6, 2023
* master: (155 commits) update readme (apache#4535) HDDS-8374. Disable flaky unit test: TestContainerStateCounts HDDS-8016. updated the ozone doc for linked bucket and deletion async limitation (apache#4526) HDDS-8237. [Snapshot] loadDb() used by SstFiltering service creates extraneous directories. (apache#4446) HDDS-8035. Intermittent timeout in TestOzoneManagerHAWithData.testOMHAMetrics (apache#4362) HDDS-8039. Allow container inspector to run from ozone debug. (apache#4337) HDDS-8304. [Snapshot] Reduce flakiness in testSkipTrackingWithZeroSnapshot (apache#4487) HDDS-7974. [Snapshot] KeyDeletingService to be aware of Ozone snapshots (apache#4486) HDDS-8368. ReplicationManager: Create ContainerReplicaOp with correct target Datanode (apache#4532) HDDS-8358. Fix the space usage comparator in ContainerBalancerSelectionCriteria (apache#4527) HDDS-8359. ReplicationManager: Fix getContainerReplicationHealth() so that it builds ContainerCheckRequest correctly (apache#4528) HDDS-8361. Useless object in TestOzoneBlockTokenIdentifier (apache#4517) HDDS-8325. Consolidate and refine RocksDB metrics of services (apache#4506) HDDS-8135. Incorrect synchronization during certificate renewal in DefaultCertificateClient. (apache#4381) HDDS-8127. Exclude deleted containers from Recon container count (apache#4440) HDDS-8364. ReadReplicas may give wrong results with topology-aware read enabled (apache#4522) HDDS-8354. Avoid WARNING about ObjectEndpoint#get (apache#4515) HDDS-8324. DN data cache gets removed randomly asking for data from disk (apache#4499) HDDS-8291. Upgrade to Hadoop 3.3.5 (apache#4484) HDDS-8355. Mark TestOMRatisSnapshots#testInstallSnapshot as flaky ...
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
The deleted container count is being fetched to subtract it from the total container count. This is because the count of deleted containers is not relevant for the current state of the cluster, and including it in the total container count would provide misleading information about the number of available containers. Therefore, it is necessary to subtract the deleted container count from the total container count to obtain the actual number of available containers. This adjusted container count is then used to construct the ClusterStateResponse that is returned by the getClusterState() method.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-8127
How was this patch tested?
Tested it manually by deleting containers from the cluster