Conversation
… not present in compaction log table (apache#9169) Reviewed-by: Swaminathan Balachandran <swamirishi.sb@gmail.com> Reviewed-by: Sadanand Shenoy <sadanand.shenoy4898@gmail.com>
…ticByteBufferPool in RpcClient (apache#9166)
… the value on SnapshotPurgeRequest (apache#9193)
…ile without sst extension (apache#9221)
… in the yaml file (apache#9224)
…ng to read cfOptions from file (apache#9225)
… lock again. (apache#9131) Co-authored-by: Sadanand Shenoy <sadanand.shenoy@cloudera.com>
…al version of snapshot (apache#9208)
Summary of ChangesHello @priyeshkaratha, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request enhances Apache Ozone by improving memory management, refining data distribution metrics, and significantly upgrading snapshot functionality. It introduces a bounded byte buffer pool to prevent memory exhaustion in client-side operations and refines container management by delegating tracking to Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces several significant enhancements and refactorings. Key changes include the introduction of a BoundedElasticByteBufferPool to prevent unbounded memory growth, and optimizations in ContainerSet to improve performance by tracking containers on a per-volume basis. A major part of this PR is the change to persist block size information in deleted block transactions, which involves modifications across the SCM and common modules, including proto definitions and the introduction of a DeletedBlock class. Additionally, there are substantial refactorings in snapshot management, particularly in OmSnapshotLocalDataManager and the checkpointing servlets, to support snapshot versioning and defragmentation. The ACL check for setTimes has also been correctly moved to preExecute for better enforcement. My main feedback is a minor correction for a copy-pasted description in an enum. Overall, the changes are substantial and appear to improve performance, resource management, and add new features.
| "for the last chunk of blocks to support HBase.)"), | ||
| WITNESSED_CONTAINER_DB_PROTO_VALUE(9, "ContainerID table schema to use value type as proto"); | ||
| WITNESSED_CONTAINER_DB_PROTO_VALUE(9, "ContainerID table schema to use value type as proto"), | ||
| STORAGE_DATA_DISTRIBUTION(10, "ContainerID table schema to use value type as proto"); |
There was a problem hiding this comment.
The description for STORAGE_DATA_DISTRIBUTION appears to be a copy-paste from the WITNESSED_CONTAINER_DB_PROTO_VALUE enum constant. Please update it to accurately describe the new layout feature, which seems to be related to persisting block sizes in deleted block transactions.
| STORAGE_DATA_DISTRIBUTION(10, "ContainerID table schema to use value type as proto"); | |
| STORAGE_DATA_DISTRIBUTION(10, "Persist block size in deleted block transactions."); |
19 participants
What changes were proposed in this pull request?
Provide a one-liner summary of the changes in the PR Title field above.
It should be in the form of
HDDS-1234. Short summary of the change.Please describe your PR in detail:
perspective not just for the reviewer.
the Jira's description if the jira is well defined.
issue investigation, github discussion, etc.
Examples of well-written pull requests:
What is the link to the Apache JIRA
Please create an issue in ASF JIRA before opening a pull request, and you need to set the title of the pull
request which starts with the corresponding JIRA issue number. (e.g. HDDS-XXXX. Fix a typo in YYY.)
(Please replace this section with the link to the Apache JIRA)
How was this patch tested?
(Please explain how this patch was tested. Ex: unit tests, manual tests, workflow run on the fork git repo.)
(If this patch involves UI changes, please attach a screenshot; otherwise, remove this.)