Skip to content

HDDS-11274. Replace Hadoop annotations/configs with Ozone-specific ones #7026

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
adoroszlai merged 2 commits into from
Aug 6, 2024
Merged

HDDS-11274. Replace Hadoop annotations/configs with Ozone-specific ones #7026

adoroszlai merged 2 commits into from
Aug 6, 2024

Conversation

@adoroszlai
Copy link
Contributor

@adoroszlai adoroszlai commented Aug 4, 2024

What changes were proposed in this pull request?

  • Replace accidental usage of Hadoop's annotations and config keys.
  • Add import restrictions to avoid similar future problems.

https://issues.apache.org/jira/browse/HDDS-11274

How was this patch tested?

Tested import restrictions by trying to compile with only the POM change:

[ERROR] Rule 2: org.apache.maven.plugins.enforcer.RestrictImports failed with message:
[ERROR] 
[ERROR] Banned imports detected:
[ERROR] 
[ERROR] Reason: Use Ozone's version of the same class
[ERROR] 	in hadoop-hdds/common/src/main/java/org/apache/hadoop/util/MutableMinMax.java
[ERROR] 		org.apache.hadoop.classification.InterfaceAudience  	(Line: 20, Matched by: org.apache.hadoop.classification.InterfaceAudience)
[ERROR] 		org.apache.hadoop.classification.InterfaceStability 	(Line: 21, Matched by: org.apache.hadoop.classification.InterfaceStability)
[ERROR] 
[ERROR] Analysis of 387 files took less than 1 second
[ERROR] 
[ERROR] Rule 3: org.apache.maven.plugins.enforcer.RestrictImports failed with message:
[ERROR] 
[ERROR] Banned imports detected:
[ERROR] 
[ERROR] Reason: Use Ozone's similar class
[ERROR] 	in hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/ratis/RatisHelper.java
[ERROR] 		org.apache.hadoop.hdfs.DFSConfigKeys 	(Line: 46, Matched by: org.apache.hadoop.hdfs.DFSConfigKeys)

CI:
https://github.com/adoroszlai/ozone/actions/runs/10234805144

@adoroszlai adoroszlai self-assigned this Aug 4, 2024
@adoroszlai adoroszlai added build Pull request that modifies the build process code-cleanup Changes that aim to make code better, without changing functionality. labels Aug 4, 2024
devmadhuu
devmadhuu reviewed Aug 6, 2024
View reviewed changes
Copy link
Contributor

@devmadhuu devmadhuu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @adoroszlai for the patch. Changes LGTM +1

@xichen01
Copy link
Contributor

xichen01 commented Aug 6, 2024

Thanks @adoroszlai for the patch. LGTM +1.

@adoroszlai adoroszlai merged commit b5cbf9b into apache:master Aug 6, 2024
@adoroszlai adoroszlai deleted the HDDS-11274 branch August 6, 2024 08:00
@adoroszlai
Copy link
Contributor Author

adoroszlai commented Aug 6, 2024

Thanks @devmadhuu, @xichen01 for the review.

errose28 added a commit to errose28/ozone that referenced this pull request Aug 7, 2024
@errose28
Merge branch 'master' into HDDS-10239-container-reconciliation
98369a8 
* master: (181 commits)
  HDDS-11289. Bump docker-maven-plugin to 0.45.0 (apache#7024)
  HDDS-11287. Code cleanup in XceiverClientSpi (apache#7043)
  HDDS-11283. Refactor KeyValueStreamDataChannel to avoid spurious IDE build issues (apache#7040)
  HDDS-11257. Ozone write does not work when http proxy is set for the JVM. (apache#7036)
  HDDS-11249. Bump ozone-runner to 20240729-jdk17-1 (apache#7003)
  HDDS-10517. Recon - Add a UI component for showing DN decommissioning detailed status and info (apache#6724)
  HDDS-11270. [hsync] Add DN layout version (HBASE_SUPPORT/version 8) upgrade test. (apache#7021)
  HDDS-11272. Statistics some node status information (apache#7025)
  HDDS-11278. Move code out of Hadoop util package (apache#7028)
  HDDS-11274. (addendum) Replace Hadoop annotations/configs with Ozone-specific ones
  HDDS-11274. Replace Hadoop annotations/configs with Ozone-specific ones (apache#7026)
  HDDS-11280. Add Synchronize in AbstractCommitWatcher.addAckDataLength (apache#7032)
  HDDS-11235. Spare InfoBucket RPC call in FileSystem#mkdir() call. (apache#6990)
  HDDS-11273. Bump commons-compress to 1.26.2 (apache#7023)
  HDDS-11225. Increase ipc.server.read.threadpool.size (apache#7007)
  HDDS-11224. Increase hdds.datanode.handler.count (apache#7011)
  HDDS-11259. [hsync] DataNode should verify HBASE_SUPPORT layout version for every PutBlock. (apache#7012)
  HDDS-11214. Added config to set rocksDB's max log file size and num of log files (apache#7014)
  HDDS-11226. Make ExponentialBackoffPolicy maxRetries configurable (apache#6985)
  HDDS-11260. [hsync] Add Ozone Manager protocol version (apache#7015)
  ...

Conflicts:
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/audit/DNAction.java
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueHandler.java
hadoop-hdds/interface-client/src/main/proto/DatanodeClientProtocol.proto
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/TestContainerReportHandler.java
errose28 added a commit to errose28/ozone that referenced this pull request Aug 13, 2024
@errose28
Merge branch 'HDDS-10239-container-reconciliation' into HDDS-11254-re…
6f68df6 
…p-supervisor

Merge conflicts are resolved but the change does not yet build.

* HDDS-10239-container-reconciliation: (183 commits)
  HDDS-10376. Add a Datanode API to supply a merkle tree for a given container. (apache#6945)
  HDDS-11289. Bump docker-maven-plugin to 0.45.0 (apache#7024)
  HDDS-11287. Code cleanup in XceiverClientSpi (apache#7043)
  HDDS-11283. Refactor KeyValueStreamDataChannel to avoid spurious IDE build issues (apache#7040)
  HDDS-11257. Ozone write does not work when http proxy is set for the JVM. (apache#7036)
  HDDS-11249. Bump ozone-runner to 20240729-jdk17-1 (apache#7003)
  HDDS-10517. Recon - Add a UI component for showing DN decommissioning detailed status and info (apache#6724)
  HDDS-10926. Block deletion should update container merkle tree. (apache#6875)
  HDDS-11270. [hsync] Add DN layout version (HBASE_SUPPORT/version 8) upgrade test. (apache#7021)
  HDDS-11272. Statistics some node status information (apache#7025)
  HDDS-11278. Move code out of Hadoop util package (apache#7028)
  HDDS-11274. (addendum) Replace Hadoop annotations/configs with Ozone-specific ones
  HDDS-11274. Replace Hadoop annotations/configs with Ozone-specific ones (apache#7026)
  HDDS-11280. Add Synchronize in AbstractCommitWatcher.addAckDataLength (apache#7032)
  HDDS-11235. Spare InfoBucket RPC call in FileSystem#mkdir() call. (apache#6990)
  HDDS-11273. Bump commons-compress to 1.26.2 (apache#7023)
  HDDS-11225. Increase ipc.server.read.threadpool.size (apache#7007)
  HDDS-11224. Increase hdds.datanode.handler.count (apache#7011)
  HDDS-11259. [hsync] DataNode should verify HBASE_SUPPORT layout version for every PutBlock. (apache#7012)
  HDDS-11214. Added config to set rocksDB's max log file size and num of log files (apache#7014)
  ...

Conflicts:
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/ozoneimpl/OzoneContainer.java
errose28 added a commit to errose28/ozone that referenced this pull request Aug 16, 2024
@errose28
Merge branch 'HDDS-10239-container-reconciliation' into HDDS-11253-co…
8601118 
…rrupt-files

* HDDS-10239-container-reconciliation: (183 commits)
  HDDS-10376. Add a Datanode API to supply a merkle tree for a given container. (apache#6945)
  HDDS-11289. Bump docker-maven-plugin to 0.45.0 (apache#7024)
  HDDS-11287. Code cleanup in XceiverClientSpi (apache#7043)
  HDDS-11283. Refactor KeyValueStreamDataChannel to avoid spurious IDE build issues (apache#7040)
  HDDS-11257. Ozone write does not work when http proxy is set for the JVM. (apache#7036)
  HDDS-11249. Bump ozone-runner to 20240729-jdk17-1 (apache#7003)
  HDDS-10517. Recon - Add a UI component for showing DN decommissioning detailed status and info (apache#6724)
  HDDS-10926. Block deletion should update container merkle tree. (apache#6875)
  HDDS-11270. [hsync] Add DN layout version (HBASE_SUPPORT/version 8) upgrade test. (apache#7021)
  HDDS-11272. Statistics some node status information (apache#7025)
  HDDS-11278. Move code out of Hadoop util package (apache#7028)
  HDDS-11274. (addendum) Replace Hadoop annotations/configs with Ozone-specific ones
  HDDS-11274. Replace Hadoop annotations/configs with Ozone-specific ones (apache#7026)
  HDDS-11280. Add Synchronize in AbstractCommitWatcher.addAckDataLength (apache#7032)
  HDDS-11235. Spare InfoBucket RPC call in FileSystem#mkdir() call. (apache#6990)
  HDDS-11273. Bump commons-compress to 1.26.2 (apache#7023)
  HDDS-11225. Increase ipc.server.read.threadpool.size (apache#7007)
  HDDS-11224. Increase hdds.datanode.handler.count (apache#7011)
  HDDS-11259. [hsync] DataNode should verify HBASE_SUPPORT layout version for every PutBlock. (apache#7012)
  HDDS-11214. Added config to set rocksDB's max log file size and num of log files (apache#7014)
  ...

Conflicts:
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/checksum/ContainerChecksumTreeManager.java
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/checksum/TestContainerChecksumTreeManager.java
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@devmadhuu devmadhuu devmadhuu left review comments

@xichen01 xichen01 Awaiting requested review from xichen01

@duongkame duongkame Awaiting requested review from duongkame

Assignees

@adoroszlai adoroszlai

Labels

build Pull request that modifies the build process code-cleanup Changes that aim to make code better, without changing functionality.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@adoroszlai @xichen01 @devmadhuu