Hdds 14010 deletion endpoint

[priyeshkaratha]

What changes were proposed in this pull request?

Provide a one-liner summary of the changes in the PR Title field above.
It should be in the form of HDDS-1234. Short summary of the change.

Please describe your PR in detail:

• What changes are proposed in the PR? and Why? It would be better if it is written from third person's
  perspective not just for the reviewer.
• Provide as much context and rationale for the pull request as possible. It could be copy-paste from
  the Jira's description if the jira is well defined.
• If it is complex code, describe the approach used to solve the issue. If possible attach design doc,
  issue investigation, github discussion, etc.

Examples of well-written pull requests:

• #3980
• #5265
• #4701
• #5283
• #5300

What is the link to the Apache JIRA

Please create an issue in ASF JIRA before opening a pull request, and you need to set the title of the pull
request which starts with the corresponding JIRA issue number. (e.g. HDDS-XXXX. Fix a typo in YYY.)

(Please replace this section with the link to the Apache JIRA)

How was this patch tested?

(Please explain how this patch was tested. Ex: unit tests, manual tests, workflow run on the fork git repo.)
(If this patch involves UI changes, please attach a screenshot; otherwise, remove this.)

[gemini-code-assist]

Summary of Changes

Hello @priyeshkaratha, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances Recon's monitoring capabilities by introducing a dedicated endpoint for tracking pending block deletions across the Ozone cluster. By aggregating deletion metrics from Ozone Manager, Storage Container Manager, and individual DataNodes, it provides a crucial overview of data lifecycle management and helps in understanding the cluster's storage utilization and health, especially during data cleanup operations.

Highlights

• New Recon Endpoint for Pending Deletions: Introduced a new REST API endpoint /api/v1/pendingDeletion in Recon, allowing users to query pending block deletion metrics from Ozone Manager (OM), Storage Container Manager (SCM), and DataNodes (DN).
• DataNode Metrics Collection Service: Implemented a DataNodeMetricsService to actively collect pending deletion metrics from individual DataNodes using their JMX endpoints, providing a consolidated view of deletion progress across the cluster.
• Configurable JMX Metric Collection: Added new configuration properties (ozone.recon.dn.metrics.collection.thread.count and ozone.recon.dn.metrics.collection.minimum.api.delay) to control the concurrency and rate-limiting of JMX metric collection from DataNodes.
• JMX Service Provider Implementation: A new JmxServiceProviderImpl was added to facilitate fetching metrics from JMX endpoints, extending Recon's capability to integrate with various metric sources.
• Enhanced Integration Tests: Updated the TestStorageDistributionEndpoint to include comprehensive integration tests for the new pending deletion endpoint, covering OM, SCM, and DataNode components, including scenarios with DataNode failures.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces a new endpoint, /api/v1/pendingDeletion, to provide metrics on pending deletions across OM, SCM, and DataNodes. The implementation includes a DataNodeMetricsService for collecting metrics from DataNodes via JMX, new configuration properties, data transfer objects, and an integration test. The changes are well-structured. My review includes feedback on a potential NullPointerException, incorrect configuration values that could lead to unexpected behavior, and an improvement for test code logging.

[priyeshkaratha]

bugbot run

[cursor]

Skipping Bugbot: Bugbot is disabled for this repository

[priyeshkaratha]

bugbot run

[cursor]

Skipping Bugbot: Bugbot is disabled for this repository