Skip to content

HDDS-2269. Provide config for fair/non-fair for OM RW Lock. #1623

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
nandakumar131 merged 2 commits into from
Oct 10, 2019
Merged

HDDS-2269. Provide config for fair/non-fair for OM RW Lock. #1623

nandakumar131 merged 2 commits into from
Oct 10, 2019

Conversation

@bharatviswa504
Copy link
Contributor

@bharatviswa504 bharatviswa504 commented Oct 9, 2019

https://issues.apache.org/jira/browse/HDDS-2269

Provide config in OzoneManager Lock for fair/non-fair for OM RW Lock.

@bharatviswa504 bharatviswa504 self-assigned this Oct 9, 2019
@hadoop-yetus
Copy link

hadoop-yetus commented Oct 9, 2019

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 42 Docker mode activated.
_ Prechecks _
+1 dupname 0 No case conflicting files found.
+1 @author 0 The patch does not contain any @author tags.
-1 test4tests 0 The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
0 mvndep 29 Maven dependency ordering for branch
-1 mvninstall 31 hadoop-hdds in trunk failed.
-1 mvninstall 32 hadoop-ozone in trunk failed.
-1 compile 23 hadoop-hdds in trunk failed.
-1 compile 17 hadoop-ozone in trunk failed.
+1 checkstyle 64 trunk passed
+1 mvnsite 1 trunk passed
+1 shadedclient 858 branch has no errors when building and testing our client artifacts.
-1 javadoc 23 hadoop-hdds in trunk failed.
-1 javadoc 18 hadoop-ozone in trunk failed.
0 spotbugs 960 Used deprecated FindBugs config; considering switching to SpotBugs.
-1 findbugs 34 hadoop-hdds in trunk failed.
-1 findbugs 21 hadoop-ozone in trunk failed.
_ Patch Compile Tests _
0 mvndep 31 Maven dependency ordering for patch
-1 mvninstall 38 hadoop-hdds in the patch failed.
-1 mvninstall 38 hadoop-ozone in the patch failed.
-1 compile 27 hadoop-hdds in the patch failed.
-1 compile 21 hadoop-ozone in the patch failed.
-1 javac 27 hadoop-hdds in the patch failed.
-1 javac 21 hadoop-ozone in the patch failed.
-0 checkstyle 31 hadoop-hdds: The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)
+1 mvnsite 0 the patch passed
+1 whitespace 0 The patch has no whitespace issues.
+1 xml 1 The patch has no ill-formed XML file.
+1 shadedclient 723 patch has no errors when building and testing our client artifacts.
-1 javadoc 20 hadoop-hdds in the patch failed.
-1 javadoc 17 hadoop-ozone in the patch failed.
-1 findbugs 34 hadoop-hdds in the patch failed.
-1 findbugs 20 hadoop-ozone in the patch failed.
_ Other Tests _
-1 unit 29 hadoop-hdds in the patch failed.
-1 unit 27 hadoop-ozone in the patch failed.
+1 asflicense 34 The patch does not generate ASF License warnings.
2444
Subsystem Report/Notes
Docker Client=19.03.3 Server=19.03.3 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/2/artifact/out/Dockerfile
GITHUB PR #1623
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml
uname Linux cd7c000a3eaa 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / 87d9f36
Default Java 1.8.0_222
mvninstall https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/2/artifact/out/branch-mvninstall-hadoop-hdds.txt
mvninstall https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/2/artifact/out/branch-mvninstall-hadoop-ozone.txt
compile https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/2/artifact/out/branch-compile-hadoop-hdds.txt
compile https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/2/artifact/out/branch-compile-hadoop-ozone.txt
javadoc https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/2/artifact/out/branch-javadoc-hadoop-hdds.txt
javadoc https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/2/artifact/out/branch-javadoc-hadoop-ozone.txt
findbugs https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/2/artifact/out/branch-findbugs-hadoop-hdds.txt
findbugs https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/2/artifact/out/branch-findbugs-hadoop-ozone.txt
mvninstall https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/2/artifact/out/patch-mvninstall-hadoop-hdds.txt
mvninstall https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/2/artifact/out/patch-mvninstall-hadoop-ozone.txt
compile https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/2/artifact/out/patch-compile-hadoop-hdds.txt
compile https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/2/artifact/out/patch-compile-hadoop-ozone.txt
javac https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/2/artifact/out/patch-compile-hadoop-hdds.txt
javac https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/2/artifact/out/patch-compile-hadoop-ozone.txt
checkstyle https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/2/artifact/out/diff-checkstyle-hadoop-hdds.txt
javadoc https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/2/artifact/out/patch-javadoc-hadoop-hdds.txt
javadoc https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/2/artifact/out/patch-javadoc-hadoop-ozone.txt
findbugs https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/2/artifact/out/patch-findbugs-hadoop-hdds.txt
findbugs https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/2/artifact/out/patch-findbugs-hadoop-ozone.txt
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/2/artifact/out/patch-unit-hadoop-hdds.txt
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/2/artifact/out/patch-unit-hadoop-ozone.txt
Test Results https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/2/testReport/
Max. process+thread count 447 (vs. ulimit of 5500)
modules C: hadoop-hdds/common hadoop-ozone/common U: .
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/2/console
versions git=2.7.4 maven=3.3.9
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

hadoop-yetus commented Oct 9, 2019

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 56 Docker mode activated.
_ Prechecks _
+1 dupname 1 No case conflicting files found.
+1 @author 0 The patch does not contain any @author tags.
-1 test4tests 0 The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
0 mvndep 70 Maven dependency ordering for branch
-1 mvninstall 37 hadoop-hdds in trunk failed.
-1 mvninstall 39 hadoop-ozone in trunk failed.
-1 compile 22 hadoop-hdds in trunk failed.
-1 compile 17 hadoop-ozone in trunk failed.
+1 checkstyle 65 trunk passed
+1 mvnsite 0 trunk passed
+1 shadedclient 854 branch has no errors when building and testing our client artifacts.
-1 javadoc 23 hadoop-hdds in trunk failed.
-1 javadoc 19 hadoop-ozone in trunk failed.
0 spotbugs 960 Used deprecated FindBugs config; considering switching to SpotBugs.
-1 findbugs 37 hadoop-hdds in trunk failed.
-1 findbugs 22 hadoop-ozone in trunk failed.
_ Patch Compile Tests _
0 mvndep 31 Maven dependency ordering for patch
-1 mvninstall 36 hadoop-hdds in the patch failed.
-1 mvninstall 40 hadoop-ozone in the patch failed.
-1 compile 28 hadoop-hdds in the patch failed.
-1 compile 21 hadoop-ozone in the patch failed.
-1 javac 28 hadoop-hdds in the patch failed.
-1 javac 21 hadoop-ozone in the patch failed.
-0 checkstyle 30 hadoop-hdds: The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)
+1 mvnsite 0 the patch passed
+1 whitespace 0 The patch has no whitespace issues.
+1 xml 2 The patch has no ill-formed XML file.
+1 shadedclient 716 patch has no errors when building and testing our client artifacts.
-1 javadoc 21 hadoop-hdds in the patch failed.
-1 javadoc 18 hadoop-ozone in the patch failed.
-1 findbugs 36 hadoop-hdds in the patch failed.
-1 findbugs 21 hadoop-ozone in the patch failed.
_ Other Tests _
-1 unit 30 hadoop-hdds in the patch failed.
-1 unit 28 hadoop-ozone in the patch failed.
+1 asflicense 36 The patch does not generate ASF License warnings.
2512
Subsystem Report/Notes
Docker Client=19.03.3 Server=19.03.3 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/1/artifact/out/Dockerfile
GITHUB PR #1623
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml
uname Linux 0f7b0cad7e0c 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / 87d9f36
Default Java 1.8.0_222
mvninstall https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/1/artifact/out/branch-mvninstall-hadoop-hdds.txt
mvninstall https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/1/artifact/out/branch-mvninstall-hadoop-ozone.txt
compile https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/1/artifact/out/branch-compile-hadoop-hdds.txt
compile https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/1/artifact/out/branch-compile-hadoop-ozone.txt
javadoc https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/1/artifact/out/branch-javadoc-hadoop-hdds.txt
javadoc https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/1/artifact/out/branch-javadoc-hadoop-ozone.txt
findbugs https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/1/artifact/out/branch-findbugs-hadoop-hdds.txt
findbugs https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/1/artifact/out/branch-findbugs-hadoop-ozone.txt
mvninstall https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/1/artifact/out/patch-mvninstall-hadoop-hdds.txt
mvninstall https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/1/artifact/out/patch-mvninstall-hadoop-ozone.txt
compile https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/1/artifact/out/patch-compile-hadoop-hdds.txt
compile https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/1/artifact/out/patch-compile-hadoop-ozone.txt
javac https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/1/artifact/out/patch-compile-hadoop-hdds.txt
javac https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/1/artifact/out/patch-compile-hadoop-ozone.txt
checkstyle https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/1/artifact/out/diff-checkstyle-hadoop-hdds.txt
javadoc https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/1/artifact/out/patch-javadoc-hadoop-hdds.txt
javadoc https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/1/artifact/out/patch-javadoc-hadoop-ozone.txt
findbugs https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/1/artifact/out/patch-findbugs-hadoop-hdds.txt
findbugs https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/1/artifact/out/patch-findbugs-hadoop-ozone.txt
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/1/artifact/out/patch-unit-hadoop-hdds.txt
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/1/artifact/out/patch-unit-hadoop-ozone.txt
Test Results https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/1/testReport/
Max. process+thread count 410 (vs. ulimit of 5500)
modules C: hadoop-hdds/common hadoop-ozone/common U: .
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/1/console
versions git=2.7.4 maven=3.3.9
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

nandakumar131
nandakumar131 requested changes Oct 9, 2019
View reviewed changes
Copy link
Contributor

@nandakumar131 nandakumar131 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall the change looks good.

hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/lock/LockManager.java Outdated
Comment on lines 30 to 32
Copy link
Contributor

@nandakumar131 nandakumar131 Oct 9, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unused imports.

Copy link
Contributor Author

@bharatviswa504 bharatviswa504 Oct 9, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/OzoneConfigKeys.java Outdated
Copy link
Contributor

@nandakumar131 nandakumar131 Oct 9, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be OZONE_MANAGER_FAIR_LOCK_DEFAULT

Copy link
Contributor Author

@bharatviswa504 bharatviswa504 Oct 9, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/lock/PooledLockFactory.java Outdated
Copy link
Contributor

@nandakumar131 nandakumar131 Oct 9, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Constructor can be package private.

Copy link
Contributor Author

@bharatviswa504 bharatviswa504 Oct 9, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@hadoop-yetus
Copy link

hadoop-yetus commented Oct 9, 2019

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 40 Docker mode activated.
_ Prechecks _
+1 dupname 1 No case conflicting files found.
+1 @author 0 The patch does not contain any @author tags.
-1 test4tests 0 The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
0 mvndep 26 Maven dependency ordering for branch
-1 mvninstall 38 hadoop-hdds in trunk failed.
-1 mvninstall 34 hadoop-ozone in trunk failed.
-1 compile 21 hadoop-hdds in trunk failed.
-1 compile 15 hadoop-ozone in trunk failed.
+1 checkstyle 51 trunk passed
+1 mvnsite 0 trunk passed
+1 shadedclient 824 branch has no errors when building and testing our client artifacts.
-1 javadoc 21 hadoop-hdds in trunk failed.
-1 javadoc 18 hadoop-ozone in trunk failed.
0 spotbugs 920 Used deprecated FindBugs config; considering switching to SpotBugs.
-1 findbugs 33 hadoop-hdds in trunk failed.
-1 findbugs 20 hadoop-ozone in trunk failed.
_ Patch Compile Tests _
0 mvndep 28 Maven dependency ordering for patch
-1 mvninstall 33 hadoop-hdds in the patch failed.
-1 mvninstall 39 hadoop-ozone in the patch failed.
-1 compile 22 hadoop-hdds in the patch failed.
-1 compile 17 hadoop-ozone in the patch failed.
-1 javac 22 hadoop-hdds in the patch failed.
-1 javac 17 hadoop-ozone in the patch failed.
+1 checkstyle 55 the patch passed
+1 mvnsite 0 the patch passed
+1 whitespace 0 The patch has no whitespace issues.
+1 xml 2 The patch has no ill-formed XML file.
+1 shadedclient 725 patch has no errors when building and testing our client artifacts.
-1 javadoc 20 hadoop-hdds in the patch failed.
-1 javadoc 18 hadoop-ozone in the patch failed.
-1 findbugs 29 hadoop-hdds in the patch failed.
-1 findbugs 19 hadoop-ozone in the patch failed.
_ Other Tests _
-1 unit 26 hadoop-hdds in the patch failed.
-1 unit 24 hadoop-ozone in the patch failed.
+1 asflicense 31 The patch does not generate ASF License warnings.
2357
Subsystem Report/Notes
Docker Client=19.03.3 Server=19.03.3 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/3/artifact/out/Dockerfile
GITHUB PR #1623
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml
uname Linux fcddccad81c5 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / 2d81abc
Default Java 1.8.0_222
mvninstall https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/3/artifact/out/branch-mvninstall-hadoop-hdds.txt
mvninstall https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/3/artifact/out/branch-mvninstall-hadoop-ozone.txt
compile https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/3/artifact/out/branch-compile-hadoop-hdds.txt
compile https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/3/artifact/out/branch-compile-hadoop-ozone.txt
javadoc https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/3/artifact/out/branch-javadoc-hadoop-hdds.txt
javadoc https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/3/artifact/out/branch-javadoc-hadoop-ozone.txt
findbugs https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/3/artifact/out/branch-findbugs-hadoop-hdds.txt
findbugs https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/3/artifact/out/branch-findbugs-hadoop-ozone.txt
mvninstall https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/3/artifact/out/patch-mvninstall-hadoop-hdds.txt
mvninstall https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/3/artifact/out/patch-mvninstall-hadoop-ozone.txt
compile https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/3/artifact/out/patch-compile-hadoop-hdds.txt
compile https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/3/artifact/out/patch-compile-hadoop-ozone.txt
javac https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/3/artifact/out/patch-compile-hadoop-hdds.txt
javac https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/3/artifact/out/patch-compile-hadoop-ozone.txt
javadoc https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/3/artifact/out/patch-javadoc-hadoop-hdds.txt
javadoc https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/3/artifact/out/patch-javadoc-hadoop-ozone.txt
findbugs https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/3/artifact/out/patch-findbugs-hadoop-hdds.txt
findbugs https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/3/artifact/out/patch-findbugs-hadoop-ozone.txt
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/3/artifact/out/patch-unit-hadoop-hdds.txt
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/3/artifact/out/patch-unit-hadoop-ozone.txt
Test Results https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/3/testReport/
Max. process+thread count 412 (vs. ulimit of 5500)
modules C: hadoop-hdds/common hadoop-ozone/common U: .
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-1623/3/console
versions git=2.7.4 maven=3.3.9
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

arp7
arp7 approved these changes Oct 9, 2019
View reviewed changes
Copy link
Contributor

@arp7 arp7 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1. Will hold off committing.

@nandakumar131 nandakumar131 merged commit 4850b3a into apache:trunk Oct 10, 2019
@nandakumar131
Copy link
Contributor

nandakumar131 commented Oct 10, 2019

Thanks @bharatviswa504 for the contribution and @arp7 for review.

amahussein pushed a commit to amahussein/hadoop that referenced this pull request Oct 29, 2019
@bharatviswa504
HDDS-2269. Provide config for fair/non-fair for OM RW Lock. (apache#1623
4dba777 
)
RogPodge pushed a commit to RogPodge/hadoop that referenced this pull request Mar 25, 2020
@bharatviswa504 @RogPodge
HDDS-2269. Provide config for fair/non-fair for OM RW Lock. (apache#1623
c6d183e 
)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@arp7 arp7 arp7 approved these changes

@nandakumar131 nandakumar131 nandakumar131 approved these changes

@anuengineer anuengineer Awaiting requested review from anuengineer

Assignees

@bharatviswa504 bharatviswa504

Labels

ozone

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@bharatviswa504 @hadoop-yetus @nandakumar131 @arp7