HDFS-16259. Catch and re-throw sub-classes of AccessControlException thrown by any permission provider plugins (eg Ranger) by sodonnel · Pull Request #3598 · apache/hadoop

sodonnel · 2021-10-28T17:38:24Z

Description of PR

When a permission provider plugin is enabled (eg Ranger) there are some scenarios where it can throw a sub-class of an AccessControlException (eg RangerAccessControlException). If this exception is allowed to propagate up the stack, it can give problems in the HDFS Client, when it unwraps the remote exception containing the AccessControlException sub-class.

Ideally, we should make AccessControlException final so it cannot be sub-classed, but that would be a breaking change at this point. Therefore I believe the safest thing to do, is to catch any AccessControlException that comes out of the permission enforcer plugin, and re-throw an AccessControlException instead.

How was this patch tested?

New Unit test

…thrown by any permission provider plugins (eg Ranger)

hadoop-yetus · 2021-10-29T01:29:02Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	1m 2s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	34m 52s		trunk passed
+1 💚	compile	1m 22s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	1m 14s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	1m 0s		trunk passed
+1 💚	mvnsite	1m 23s		trunk passed
+1 💚	javadoc	0m 56s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 27s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	3m 18s		trunk passed
+1 💚	shadedclient	25m 13s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 11s		the patch passed
+1 💚	compile	1m 16s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
-1 ❌	javac	1m 16s	/results-compile-javac-hadoop-hdfs-project_hadoop-hdfs-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.txt	hadoop-hdfs-project_hadoop-hdfs-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 generated 1 new + 460 unchanged - 1 fixed = 461 total (was 461)
+1 💚	compile	1m 10s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
-1 ❌	javac	1m 10s	/results-compile-javac-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt	hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~~20.04-b10 with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~~20.04-b10 generated 1 new + 444 unchanged - 1 fixed = 445 total (was 445)
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 53s		the patch passed
+1 💚	mvnsite	1m 13s		the patch passed
+1 💚	javadoc	0m 48s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 20s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	3m 23s		the patch passed
+1 💚	shadedclient	24m 53s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	363m 0s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 59s		The patch does not generate ASF License warnings.
		469m 20s

Reason	Tests
Failed junit tests	hadoop.security.TestPermissionSymlinks

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3598/1/artifact/out/Dockerfile
GITHUB PR	#3598
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 1a812d30d785 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `43cca9c`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3598/1/testReport/
Max. process+thread count	1798 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3598/1/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

jojochuang

LGTM. thank you

sodonnel · 2021-10-29T08:06:18Z

There is a related test failure in TestPermissionSymlink.testAccess, as it actually throws a TraverseAccessControlException, which is being caught and then re-thrown as an AccessControlException.

hadoop-yetus · 2021-10-30T19:11:21Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 42s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	32m 16s		trunk passed
+1 💚	compile	1m 21s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	1m 17s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	1m 0s		trunk passed
+1 💚	mvnsite	1m 23s		trunk passed
+1 💚	javadoc	0m 59s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 29s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	3m 7s		trunk passed
+1 💚	shadedclient	21m 56s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 9s		the patch passed
+1 💚	compile	1m 12s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
-1 ❌	javac	1m 12s	/results-compile-javac-hadoop-hdfs-project_hadoop-hdfs-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.txt	hadoop-hdfs-project_hadoop-hdfs-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 generated 1 new + 460 unchanged - 1 fixed = 461 total (was 461)
+1 💚	compile	1m 8s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
-1 ❌	javac	1m 8s	/results-compile-javac-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt	hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~~20.04-b10 with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~~20.04-b10 generated 1 new + 444 unchanged - 1 fixed = 445 total (was 445)
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 52s		the patch passed
+1 💚	mvnsite	1m 15s		the patch passed
+1 💚	javadoc	0m 46s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 22s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	3m 9s		the patch passed
+1 💚	shadedclient	21m 36s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	223m 50s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 45s		The patch does not generate ASF License warnings.
		320m 19s

Reason	Tests
Failed junit tests	hadoop.hdfs.TestRollingUpgrade

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3598/2/artifact/out/Dockerfile
GITHUB PR	#3598
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux c20c1be1125b 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `0d59245`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3598/2/testReport/
Max. process+thread count	3251 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3598/2/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

sodonnel · 2021-10-30T19:27:16Z

The two failing unit tests pass locally:

org.apache.hadoop.hdfs.TestRollingUpgrade.testDFSAdminRollingUpgradeCommands
org.apache.hadoop.hdfs.TestRollingUpgrade.testRollback

The javac warnings are because I added a try block around an already deprecated method, so I don't think I can avoid them.

jojochuang

+1 thank you.

…thrown by any permission provider plugins (eg Ranger) (#3598) (cherry picked from commit 2f35cc3) Conflicts: hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestINodeAttributeProvider.java

…thrown by any permission provider plugins (eg Ranger) (apache#3598)

HDFS-16529. Catch and re-throw sub-classes of AccessControlException …

43cca9c

…thrown by any permission provider plugins (eg Ranger)

jojochuang approved these changes Oct 29, 2021

View reviewed changes

sodonnel changed the title ~~HDFS-16529. Catch and re-throw sub-classes of AccessControlException thrown by any permission provider plugins (eg Ranger)~~ HDFS-16259. Catch and re-throw sub-classes of AccessControlException thrown by any permission provider plugins (eg Ranger) Oct 29, 2021

Also handle TraverseAccessControlException

0d59245

jojochuang approved these changes Nov 2, 2021

View reviewed changes

sodonnel merged commit 2f35cc3 into apache:trunk Nov 2, 2021

HarshitGupta11 pushed a commit to HarshitGupta11/hadoop that referenced this pull request Nov 28, 2022

HDFS-16259. Catch and re-throw sub-classes of AccessControlException …

c8400dc

…thrown by any permission provider plugins (eg Ranger) (apache#3598)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HDFS-16259. Catch and re-throw sub-classes of AccessControlException thrown by any permission provider plugins (eg Ranger)#3598

HDFS-16259. Catch and re-throw sub-classes of AccessControlException thrown by any permission provider plugins (eg Ranger)#3598
sodonnel merged 2 commits intoapache:trunkfrom
sodonnel:HDFS-16529

sodonnel commented Oct 28, 2021

Uh oh!

hadoop-yetus commented Oct 29, 2021

Uh oh!

jojochuang left a comment

Uh oh!

sodonnel commented Oct 29, 2021

Uh oh!

hadoop-yetus commented Oct 30, 2021

Uh oh!

sodonnel commented Oct 30, 2021

Uh oh!

jojochuang left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

sodonnel commented Oct 28, 2021

Description of PR

How was this patch tested?

Uh oh!

hadoop-yetus commented Oct 29, 2021

Uh oh!

jojochuang left a comment

Choose a reason for hiding this comment

Uh oh!

sodonnel commented Oct 29, 2021

Uh oh!

hadoop-yetus commented Oct 30, 2021

Uh oh!

sodonnel commented Oct 30, 2021

Uh oh!

jojochuang left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants