HDDS-13445. Make ozone debug replicas chunk-info stream json output between datanode calls
#8914
+146
−104
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
|
@Gargi-jais11 the PR description does not seem to match the actual change, and the branch references HDDS-13445, not HDDS-12998 |
Gargi-jais11
changed the title
ozone debug replicas chunk-info stream json output between datanode calls
Contributor
Author
So sorry. I have changed PR description. |
errose28
reviewed
Aug 7, 2025
...-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/replicas/chunk/ChunkKeyHandler.java
Outdated
Show resolved
Hide resolved
Gargi-jais11
force-pushed
the
HDDS-13445
branch
from
7dc7c36 to
03eaad3
Compare
errose28
reviewed
Aug 19, 2025
Contributor
errose28
left a comment
errose28
left a comment
There was a problem hiding this comment.
Thanks for the improvement. Mostly looks good. I tested it manually as well. Just two minor comments.
...-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/replicas/chunk/ChunkKeyHandler.java
Show resolved
Hide resolved
sarvekshayr
reviewed
Aug 20, 2025
Contributor
sarvekshayr
left a comment
sarvekshayr
left a comment
There was a problem hiding this comment.
Tested locally and verified the streaming output. Other than Ethan’s comments, overall LGTM.
Contributor
|
Thanks @Gargi-jais11 for the patch, and @errose28 @sarvekshayr for the reviews |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
ozone debug replicas chunk-info prints chunk information from all replicas for all chunks of all blocks within a file. It gathers all the information from the datanodes, stores it in memory, then prints it all at once. For large files in the GB range, this could result in a large amount of information stored in the client memory before printing. It would be better to print information about one block at a time in between each getBlock call to the datanode. The Json structure can remain the same.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-13445
How was this patch tested?
ran manually on docker-cluster