Skip to content

HDFS-16616. remove use of org.apache.hadoop.util.Sets#4400

Merged
PrabhuJoseph merged 1 commit intoapache:trunkfrom
Samrat002:HDFS-16616
Jun 22, 2022
Merged

HDFS-16616. remove use of org.apache.hadoop.util.Sets#4400
PrabhuJoseph merged 1 commit intoapache:trunkfrom
Samrat002:HDFS-16616

Conversation

@Samrat002
Copy link
Contributor

@Samrat002 Samrat002 commented Jun 3, 2022

Description of PR

https://issues.apache.org/jira/browse/HDFS-16616

As part of removing guava dependencies HADOOP-17115, HADOOP-17721, HADOOP-17722 and HADOOP-17720 are fixed,

Currently the code call util function to create HashSet and TreeSet in the repo . These function calls dont have much importance as it is calling internally new HashSet<> / new TreeSet<> from java.utils

This task is to clean up all the function calls to create sets which is redundant

Before moving to java8 , sets were created using guava functions and API , now since this is moved away and util code in the hadoop now looks like

public static <E extends Comparable> TreeSet<E> newTreeSet() { return new TreeSet<E>();}
public static <E> HashSet<E> newHashSet()

{ return new HashSet(); }

These interfaces dont do anything much just a extra layer of function call

please refer to the task
https://issues.apache.org/jira/browse/HADOOP-17726

Can anyone review if this ticket add some value in the code.
Looking forward to some input/ thoughts . If not adding any value we can close it and not move forward with changes !

How was this patch tested?

For code changes:

  • Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?

@hadoop-yetus
Copy link

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 17m 56s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 1s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 10 new or modified test files.
_ trunk Compile Tests _
+0 🆗 mvndep 14m 30s Maven dependency ordering for branch
+1 💚 mvninstall 24m 46s trunk passed
+1 💚 compile 6m 20s trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚 compile 6m 1s trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚 checkstyle 1m 40s trunk passed
+1 💚 mvnsite 2m 49s trunk passed
+1 💚 javadoc 2m 25s trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚 javadoc 3m 6s trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚 spotbugs 5m 16s trunk passed
+1 💚 shadedclient 20m 58s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 1m 3s Maven dependency ordering for patch
+1 💚 mvninstall 2m 11s the patch passed
+1 💚 compile 6m 9s the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚 javac 6m 9s the patch passed
+1 💚 compile 5m 50s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚 javac 5m 50s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 1m 22s hadoop-hdfs-project: The patch generated 0 new + 385 unchanged - 1 fixed = 385 total (was 386)
+1 💚 mvnsite 2m 12s the patch passed
+1 💚 javadoc 1m 45s the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚 javadoc 2m 39s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚 spotbugs 4m 51s the patch passed
+1 💚 shadedclient 20m 43s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 398m 10s hadoop-hdfs in the patch passed.
+1 💚 unit 35m 8s hadoop-hdfs-rbf in the patch passed.
+1 💚 asflicense 1m 17s The patch does not generate ASF License warnings.
592m 1s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4400/1/artifact/out/Dockerfile
GITHUB PR #4400
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux 8c9263be3d0c 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 6d3e3c984a0626448845490c62d6fb3fce59e897
Default Java Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4400/1/testReport/
Max. process+thread count 2794 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4400/1/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

remove non required removal of null checks and sanitory changes
@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 1m 4s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 1s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 10 new or modified test files.
_ trunk Compile Tests _
+0 🆗 mvndep 14m 27s Maven dependency ordering for branch
+1 💚 mvninstall 27m 37s trunk passed
+1 💚 compile 7m 17s trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚 compile 6m 30s trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚 checkstyle 1m 36s trunk passed
+1 💚 mvnsite 2m 37s trunk passed
-1 ❌ javadoc 1m 22s /branch-javadoc-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt hadoop-hdfs in trunk failed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.
+1 💚 javadoc 2m 54s trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚 spotbugs 5m 59s trunk passed
+1 💚 shadedclient 25m 36s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 51s Maven dependency ordering for patch
+1 💚 mvninstall 2m 19s the patch passed
+1 💚 compile 8m 48s the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚 javac 8m 48s the patch passed
+1 💚 compile 7m 1s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚 javac 7m 1s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 1m 22s hadoop-hdfs-project: The patch generated 0 new + 385 unchanged - 1 fixed = 385 total (was 386)
+1 💚 mvnsite 2m 24s the patch passed
-1 ❌ javadoc 1m 10s /patch-javadoc-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt hadoop-hdfs in the patch failed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.
+1 💚 javadoc 2m 39s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚 spotbugs 5m 18s the patch passed
+1 💚 shadedclient 24m 29s patch has no errors when building and testing our client artifacts.
_ Other Tests _
-1 ❌ unit 457m 48s /patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt hadoop-hdfs in the patch passed.
+1 💚 unit 39m 0s hadoop-hdfs-rbf in the patch passed.
+1 💚 asflicense 1m 20s The patch does not generate ASF License warnings.
656m 26s
Reason Tests
Failed junit tests hadoop.hdfs.server.balancer.TestBalancerWithHANameNodes
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4400/2/artifact/out/Dockerfile
GITHUB PR #4400
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux 7a27e22e00a2 4.15.0-175-generic #184-Ubuntu SMP Thu Mar 24 17:48:36 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 8aecdf08dc032fc7297a8c6bd65e7899f63712e4
Default Java Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4400/2/testReport/
Max. process+thread count 2669 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4400/2/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 1m 22s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+0 🆗 detsecrets 0m 1s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 10 new or modified test files.
_ trunk Compile Tests _
+0 🆗 mvndep 15m 8s Maven dependency ordering for branch
+1 💚 mvninstall 29m 32s trunk passed
+1 💚 compile 7m 29s trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚 compile 8m 38s trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚 checkstyle 1m 42s trunk passed
+1 💚 mvnsite 3m 0s trunk passed
-1 ❌ javadoc 2m 1s /branch-javadoc-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt hadoop-hdfs in trunk failed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.
+1 💚 javadoc 2m 56s trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚 spotbugs 5m 57s trunk passed
+1 💚 shadedclient 24m 5s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 29s Maven dependency ordering for patch
+1 💚 mvninstall 2m 13s the patch passed
+1 💚 compile 7m 27s the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚 javac 7m 27s the patch passed
+1 💚 compile 7m 28s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚 javac 7m 28s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 1m 35s hadoop-hdfs-project: The patch generated 0 new + 385 unchanged - 1 fixed = 385 total (was 386)
+1 💚 mvnsite 2m 47s the patch passed
-1 ❌ javadoc 1m 18s /patch-javadoc-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt hadoop-hdfs in the patch failed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.
+1 💚 javadoc 2m 42s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚 spotbugs 5m 57s the patch passed
+1 💚 shadedclient 26m 52s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 456m 39s hadoop-hdfs in the patch passed.
+1 💚 unit 37m 7s hadoop-hdfs-rbf in the patch passed.
+1 💚 asflicense 1m 18s The patch does not generate ASF License warnings.
660m 25s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4400/3/artifact/out/Dockerfile
GITHUB PR #4400
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux 1a3d66fc3631 4.15.0-175-generic #184-Ubuntu SMP Thu Mar 24 17:48:36 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 856d72f
Default Java Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4400/3/testReport/
Max. process+thread count 2322 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4400/3/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@Samrat002
Copy link
Contributor Author

Samrat002 commented Jun 16, 2022

  • hadoop-hdfs in trunk failed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1. failed for trunk and the patch
  • All Test passed .
    Please review the pr
    @PrabhuJoseph
    Thank you

@PrabhuJoseph
Copy link
Contributor

Thanks @Samrat002 for the patch. Looks good to me. WIll commit it tomorrow morning if no other comments.

@PrabhuJoseph PrabhuJoseph merged commit e8fd914 into apache:trunk Jun 22, 2022
HarshitGupta11 pushed a commit to HarshitGupta11/hadoop that referenced this pull request Nov 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

Comments