Skip to content

Conversation

@sadanand48
Copy link
Contributor

What changes were proposed in this pull request?

Main changes

  1. Make org.apache.hadoop.ozone.snapshot.SnapshotDiffReport extend org.apache.hadoop.hdfs.protocol.SnapshotDiffReport
  2. Add Ozone's DistcpSync implementation that provides flexibility to use Ozone specific checks/ functions.

Changes to pom.xml

  1. added hadoop-distcp dependency to ozonefs-common
  2. added hadoop-hdfs-client to ozone-common (required to access org.apache.hadoop.hdfs.protocol.SnapshotDiffReport)
  3. update hadoop version to 3.4.0-Snapshot that contains changes for distcp to work with ozone with snapshots.
  4. Had to update kerby.version as it was throwing Dependency convergence error for org.apache.kerby:kerb-core:jar:1.0.1 since latest version of hadoop uses this version.

this change requires latest version of hadoop that contains HDFS-16911.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-7916

How was this patch tested?

Unit tests

@sadanand48 sadanand48 changed the title Distcp with snapshots Ozone change HDDS-7916. [Snapshot] Support for Distcp with Ozone Snapshots. Feb 7, 2023
@aswinshakil aswinshakil added the snapshot https://issues.apache.org/jira/browse/HDDS-6517 label Feb 27, 2023
@prashantpogde
Copy link
Contributor

@sadanand48 Can you please resolve the conflicts ?

@Override
public void readUnlock(String opName) {
// not used anywhere, implemented as part of interface.
this.lock.writeLock().lock();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

writeLock ??

Copy link
Contributor

@prashantpogde prashantpogde left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other changes look good to me.

@smengcl
Copy link
Contributor

smengcl commented Mar 17, 2023

Thanks @sadanand48 . Can we rebase this and mark this as ready for review?

@smengcl smengcl requested a review from jojochuang March 17, 2023 05:50
@smengcl smengcl marked this pull request as ready for review March 28, 2023 18:22
@sadanand48
Copy link
Contributor Author

Broke down this patch into a change which doesn't depend on hdfs version upgrade and created #4491

@smengcl
Copy link
Contributor

smengcl commented Mar 31, 2023

Broke down this patch into a change which doesn't depend on hdfs version upgrade and created #4491

Thanks @sadanand48 . Shall we close this PR then?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

snapshot https://issues.apache.org/jira/browse/HDDS-6517

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants