Skip to content

HDDS-12984. Use InodeID to identify the SST files inside the tarball. #8477

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
jojochuang merged 12 commits into from
Jun 25, 2025
Merged

HDDS-12984. Use InodeID to identify the SST files inside the tarball. #8477

jojochuang merged 12 commits into from
Jun 25, 2025

Conversation

@sadanand48
Copy link
Contributor

@sadanand48 sadanand48 commented May 19, 2025

What changes were proposed in this pull request?

Use InodeID to identify the SST files inside the tarball : Add Inode named files inside the tarball

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-12984

How was this patch tested?

Unit test

swamirishi
swamirishi requested changes May 19, 2025
View reviewed changes
Copy link
Contributor

@swamirishi swamirishi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the patch @sadanand48 I have taken an overall look at the code changes I see we are trying to force fit to the previous algo which doesn't necessarily have to be the case here. I have left comments inline for the changes required.

@swamirishi
Copy link
Contributor

swamirishi commented May 19, 2025

Let us copy the code for this implementation, since the algorithm might change significantly and in order to keep the implementation clean it is ok to have some intersection.

@jojochuang jojochuang added the snapshot https://issues.apache.org/jira/browse/HDDS-6517 label May 19, 2025
swamirishi
swamirishi requested changes Jun 6, 2025
View reviewed changes
Copy link
Contributor

@swamirishi swamirishi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks @sadanand48 for addressing the previous review comments left few more review comments inline

@swamirishi swamirishi mentioned this pull request Jun 8, 2025
Merged
@jojochuang jojochuang requested a review from swamirishi June 10, 2025 22:25
jojochuang
jojochuang reviewed Jun 13, 2025
View reviewed changes
Copy link
Contributor

@jojochuang jojochuang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ideally public method should all have test coverage.

@sadanand48 sadanand48 marked this pull request as ready for review June 18, 2025 18:05
@jojochuang jojochuang requested a review from swamirishi June 23, 2025 22:58
jojochuang
jojochuang approved these changes Jun 23, 2025
View reviewed changes
Copy link
Contributor

@jojochuang jojochuang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking good to me.

@jojochuang jojochuang merged commit 96390ac into apache:master Jun 25, 2025
79 of 80 checks passed
@peterxcli
Copy link
Member

peterxcli commented Jun 25, 2025

Is the new class OMDBCheckpointServletInodeBasedXfer used in production code? Cause I dont see any reference of it?

@sadanand48
Copy link
Contributor Author

sadanand48 commented Jun 25, 2025

@peterxcli not as of now, I will add it in follow up PR's.

errose28 added a commit to errose28/ozone that referenced this pull request Jun 26, 2025
@errose28
Merge branch 'master' into diff-report-javadoc
8b9c416 
* master: (90 commits)
  HDDS-12468. Check for space availability for all dns while container creation in pipeline (apache#8663)
  HDDS-12151. Fail write when volume is full considering min free space (apache#8642)
  HDDS-13251. Update Byteman usage README license (apache#8700)
  HDDS-13251. Support dynamic Byteman scripts via bmsubmit in ozonesecure-ha (apache#8654)
  HDDS-12070. Bump Ratis to 3.2.0 (apache#8689)
  HDDS-12984. Use InodeID to identify the SST files inside the tarball. (apache#8477)
  HDDS-13319. Simplify KeyPrefixFilter (apache#8692)
  HDDS-13295. Remove jackson1 exclusions for hadoop-common (apache#8687)
  HDDS-13314. Remove unused maven-pdf-plugin (apache#8686)
  HDDS-13324. Optimize memory footprint for Recon listKeys API (apache#8680)
  HDDS-13240. Add newly added metrics into grafana dashboard. (apache#8656)
  HDDS-13309. Add keyIterator/valueIterator methods to Table. (apache#8675)
  HDDS-13325. Introduce OZONE_SERVER_OPTS for common options for server processes (apache#8685)
  HDDS-13318. Simplify the getRangeKVs methods in Table (apache#8683)
  HDDS-13322. Remove module auto-detection from flaky-test-check (apache#8679)
  HDDS-13289. Remove usage of Jetty StringUtil (apache#8684)
  HDDS-13270. Reduce getBucket API invocations in S3 bucket owner verification (apache#8653)
  HDDS-13288. Container checksum file proto changes to account for deleted blocks. (apache#8649)
  HDDS-13306. Intermittent failure in testDirectoryDeletingServiceIntervalReconfiguration (apache#8682)
  HDDS-13317. Table should support empty array/String (apache#8676)
  ...
errose28 added a commit to errose28/ozone that referenced this pull request Jun 26, 2025
@errose28
Merge branch 'master' into HDDS-13083-handle-empty-mt
b248828 
* master: (170 commits)
  HDDS-12468. Check for space availability for all dns while container creation in pipeline (apache#8663)
  HDDS-12151. Fail write when volume is full considering min free space (apache#8642)
  HDDS-13251. Update Byteman usage README license (apache#8700)
  HDDS-13251. Support dynamic Byteman scripts via bmsubmit in ozonesecure-ha (apache#8654)
  HDDS-12070. Bump Ratis to 3.2.0 (apache#8689)
  HDDS-12984. Use InodeID to identify the SST files inside the tarball. (apache#8477)
  HDDS-13319. Simplify KeyPrefixFilter (apache#8692)
  HDDS-13295. Remove jackson1 exclusions for hadoop-common (apache#8687)
  HDDS-13314. Remove unused maven-pdf-plugin (apache#8686)
  HDDS-13324. Optimize memory footprint for Recon listKeys API (apache#8680)
  HDDS-13240. Add newly added metrics into grafana dashboard. (apache#8656)
  HDDS-13309. Add keyIterator/valueIterator methods to Table. (apache#8675)
  HDDS-13325. Introduce OZONE_SERVER_OPTS for common options for server processes (apache#8685)
  HDDS-13318. Simplify the getRangeKVs methods in Table (apache#8683)
  HDDS-13322. Remove module auto-detection from flaky-test-check (apache#8679)
  HDDS-13289. Remove usage of Jetty StringUtil (apache#8684)
  HDDS-13270. Reduce getBucket API invocations in S3 bucket owner verification (apache#8653)
  HDDS-13288. Container checksum file proto changes to account for deleted blocks. (apache#8649)
  HDDS-13306. Intermittent failure in testDirectoryDeletingServiceIntervalReconfiguration (apache#8682)
  HDDS-13317. Table should support empty array/String (apache#8676)
  ...

Conflicts:
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/checksum/ContainerChecksumTreeManager.java
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/checksum/TestContainerChecksumTreeManager.java
jojochuang added a commit to jojochuang/ozone that referenced this pull request Nov 19, 2025
@jojochuang
Revert "HDDS-12984. Use InodeID to identify the SST files inside the …
e5f2d99 
…tarball. (apache#8477)"

This reverts commit 96390ac.

 Conflicts:
	hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOMDbCheckpointServletInodeBasedXfer.java
	hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OMDBCheckpointServletInodeBasedXfer.java

Change-Id: I777c0c8145830df85098afde7b89ff7864472534
jojochuang added a commit to jojochuang/ozone that referenced this pull request Nov 20, 2025
@jojochuang
Revert "HDDS-12984. Use InodeID to identify the SST files inside the …
c6b6356 
…tarball. (apache#8477)"

This reverts commit 96390ac.

 Conflicts:
	hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOMDbCheckpointServletInodeBasedXfer.java
	hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OMDBCheckpointServletInodeBasedXfer.java

Change-Id: I777c0c8145830df85098afde7b89ff7864472534
@jojochuang jojochuang mentioned this pull request Nov 20, 2025
Closed
jojochuang added a commit that referenced this pull request Nov 21, 2025
@jojochuang
Revert "HDDS-12984. Use InodeID to identify the SST files inside the …
3ca52fd 
…tarball. (#8477)"

This reverts commit 96390ac.

 Conflicts:
	hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOMDbCheckpointServletInodeBasedXfer.java
	hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OMDBCheckpointServletInodeBasedXfer.java

Change-Id: I777c0c8145830df85098afde7b89ff7864472534
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jojochuang jojochuang jojochuang approved these changes

@swamirishi swamirishi Awaiting requested review from swamirishi

+1 more reviewer

@SaketaChalamchala SaketaChalamchala SaketaChalamchala left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

snapshot https://issues.apache.org/jira/browse/HDDS-6517

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@sadanand48 @swamirishi @peterxcli @jojochuang @SaketaChalamchala