Skip to content

HDDS-8339. Recon Show the number of keys marked for Deletion in Recon UI. #4519

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
dombizita merged 1 commit into from
May 15, 2023
Merged

HDDS-8339. Recon Show the number of keys marked for Deletion in Recon UI. #4519

dombizita merged 1 commit into from
May 15, 2023

Conversation

@smitajoshi12
Copy link
Contributor

@smitajoshi12 smitajoshi12 commented Apr 3, 2023 •
edited
Loading

What changes were proposed in this pull request?

This information will be displayed on the overview page of recon which gets populated with the information provided by the ClusterStateEndpoint This information can be useful for auditing, compliance, and troubleshooting purposes. Hence in short the main purpose is to get a quick idea of the deletion progress.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-8339

How was this patch tested?

Manually

image

@smitajoshi12 smitajoshi12 changed the title HDDS-8399. Show the number of keys marked for Deletion in Recon UI HDDS-8399. Recon Show the number of keys marked for Deletion in Recon UI. Apr 3, 2023
@ArafatKhan2198
Copy link
Contributor

ArafatKhan2198 commented Apr 3, 2023

@smitajoshi12 Instead of naming the tile "Number of Deleted Keys," let's call it "Pending key Deletions" as this number represents the keys marked for deletion. They will be deleted eventually by the KeyDeletionService task running in the background for SCM.

@ArafatKhan2198
Copy link
Contributor

ArafatKhan2198 commented Apr 3, 2023

@smitajoshi12, please position the "Deleted Keys" tile after the "Keys" tile, allowing users to easily compare both numbers simultaneously.

@kerneltime kerneltime requested a review from dombizita April 3, 2023 16:29
@kerneltime
Copy link
Contributor

kerneltime commented Apr 3, 2023

@devmadhuu can you please take a look?

dombizita
dombizita reviewed Apr 4, 2023
View reviewed changes
Copy link
Contributor

@dombizita dombizita left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for working on this @smitajoshi12! I left a few comments inline, please take a look

@smitajoshi12
Copy link
Contributor Author

smitajoshi12 commented Apr 5, 2023

@smitajoshi12, please position the "Deleted Keys" tile after the "Keys" tile, allowing users to easily compare both numbers simultaneously.

Done

@ArafatKhan2198 ArafatKhan2198 mentioned this pull request Apr 12, 2023
Merged
@smitajoshi12 smitajoshi12 changed the title HDDS-8399. Recon Show the number of keys marked for Deletion in Recon UI. HDDS-8339. Recon Show the number of keys marked for Deletion in Recon UI. Apr 14, 2023
@dombizita
Copy link
Contributor

dombizita commented Apr 14, 2023

thanks for updating your patch @smitajoshi12, now for me with this check it is not showing 0 in case there is no key/bucket/pending delete key. I attached a screenshot (I did a docker cluster locally with your changes), please check it.

recon

@smitajoshi12
Copy link
Contributor Author

smitajoshi12 commented Apr 14, 2023 •
edited
Loading

@dombizita
Thanks Zita completed changes suggested by you currently there are no changes integrated for pendingDeleteKeys in clusterState API so coming empty buckets.

image

dombizita
dombizita approved these changes Apr 28, 2023
View reviewed changes
Copy link
Contributor

@dombizita dombizita left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for working on this @smitajoshi12, it looks good to me!

@dombizita dombizita requested a review from smengcl April 28, 2023 11:02
@smitajoshi12
HDDS-8339. Recon Show the number of keys marked for Deletion in Recon…
bdcc877 
… UI. Revert Back keysPendingDeletion key chnages
smengcl
smengcl approved these changes May 5, 2023
View reviewed changes
Copy link
Contributor

@smengcl smengcl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 pending CI

@ArafatKhan2198
Copy link
Contributor

ArafatKhan2198 commented May 9, 2023

@smitajoshi12 can you please check the UT's that are failing.

@smitajoshi12
Copy link
Contributor Author

smitajoshi12 commented May 9, 2023

@smitajoshi12 can you please check the UT's that are failing.

@dombizita @smengcl @devmadhuu
Hi Zita , In Local build issuccessful can you rerun

@dombizita dombizita merged commit 257a64f into apache:master May 15, 2023
@dombizita
Copy link
Contributor

dombizita commented May 15, 2023

thanks for working on this patch @smitajoshi12! thanks for the review @ArafatKhan2198, @devmadhuu and @smengcl!

errose28 added a commit to errose28/ozone that referenced this pull request May 17, 2023
@errose28
Merge branch 'master' into HDDS-7366
fb44948 
* master: (78 commits)
  HDDS-8575. Intermittent failure in TestCloseContainerEventHandler.testCloseContainerWithDelayByLeaseManager (apache#4688)
  HDDS-7241. EC: Reconstruction could fail with orphan blocks. (apache#4718)
  HDDS-8577. [Snapshot] Disable compaction log when loading metadata for snapshot (apache#4697)
  HDDS-7080. EC: Offline reconstruction needs better logging (apache#4719)
  HDDS-8626. Config thread pool in ReplicationServer (apache#4715)
  HDDS-8616. Underreplication not fixed if all replicas start decommissioning (apache#4711)
  HDDS-8254. Close containers when volume reaches utilisation threshold (apache#4583)
  HDDS-8254. Close containers when volume reaches utilisation threshold (apache#4583)
  HDDS-8615. Explicitly show EC block type in 'ozone debug chunkinfo' command output (apache#4706)
  HDDS-8623. Delete duplicate getBucketInfo in OMKeyCommitRequest (apache#4712)
  HDDS-8339. Recon Show the number of keys marked for Deletion in Recon UI. (apache#4519)
  HDDS-8572. Support CodecBuffer for protobuf v3 codecs. (apache#4693)
  HDDS-8010. Improve DN warning message when getBlock does not find the block. (apache#4698)
  HDDS-8621. IOException is never thrown in SCMRatisServer.getRatisRoles(). (apache#4710)
  HDDS-8463. S3 key uniqueness in deletedTable (apache#4660)
  HDDS-8584. Hadoop client write slowly when stream enabled (apache#4703)
  HDDS-7732. EC: Verify block deletion from missing EC containers (apache#4705)
  HDDS-8581. Avoid random ports in integration tests (apache#4699)
  HDDS-8504. ReplicationManager: Pass used and excluded node separately for Under and Mis-Replication (apache#4694)
  HDDS-8576. Close RocksDB instance in RDBStore if RDBStore's initialization fails after RocksDB instance creation (apache#4692)
  ...
@smitajoshi12 smitajoshi12 deleted the HDDS-8339 branch May 22, 2023 10:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@devmadhuu devmadhuu devmadhuu left review comments

@smengcl smengcl smengcl approved these changes

@dombizita dombizita dombizita approved these changes

@ArafatKhan2198 ArafatKhan2198 ArafatKhan2198 approved these changes

Assignees

No one assigned

Labels

recon

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@smitajoshi12 @ArafatKhan2198 @kerneltime @dombizita @devmadhuu @smengcl @adoroszlai