HDFS-15561. RBF: Remove NPE when local namenode is not configured #2954

fengnanli · 2021-04-25T06:19:40Z

NOTICE

Please create an issue in ASF JIRA before opening a pull request,
and you need to set the title of the pull request which starts with
the corresponding JIRA issue number. (e.g. HADOOP-XXXXX. Fix a typo in YYY.)
For more details, please see https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute

hadoop-yetus · 2021-04-25T10:08:55Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	13m 32s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	markdownlint	0m 0s		markdownlint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 3 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	15m 40s		Maven dependency ordering for branch
+1 💚	mvninstall	20m 28s		trunk passed
+1 💚	compile	20m 59s		trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	compile	18m 0s		trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	checkstyle	3m 48s		trunk passed
+1 💚	mvnsite	2m 27s		trunk passed
+1 💚	javadoc	1m 57s		trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	2m 47s		trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	spotbugs	3m 44s		trunk passed
+1 💚	shadedclient	14m 35s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 28s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 26s		the patch passed
+1 💚	compile	20m 2s		the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javac	20m 2s		the patch passed
+1 💚	compile	18m 8s		the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	javac	18m 8s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	3m 46s		the patch passed
+1 💚	mvnsite	2m 23s		the patch passed
+1 💚	javadoc	1m 57s		the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	2m 49s		the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	spotbugs	4m 7s		the patch passed
+1 💚	shadedclient	15m 1s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	17m 34s		hadoop-common in the patch passed.
+1 💚	unit	18m 12s		hadoop-hdfs-rbf in the patch passed.
+1 💚	asflicense	0m 57s		The patch does not generate ASF License warnings.
		228m 3s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2954/1/artifact/out/Dockerfile
GITHUB PR	#2954
Optional Tests	dupname asflicense mvnsite codespell markdownlint compile javac javadoc mvninstall unit shadedclient spotbugs checkstyle
uname	Linux ceadae4ebf44 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 4d83dc44ee06d94d12dc0d648be906d7df2072fb
Default Java	Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2954/1/testReport/
Max. process+thread count	3310 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-rbf U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2954/1/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

Hexiaoqiao

Thanks @fengnanli for your works. It makes sense to me about to fix NPE for local namenode.
I find it involves some metric improvement. IMO it is better to split it to another PR to do that. Thanks.

fengnanli · 2021-04-26T18:12:22Z

Thanks for the review @Hexiaoqiao I accidentally checked out this branch from another WIP branch. Rebase it on the latest trunk now.

hadoop-yetus · 2021-04-26T19:46:16Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 37s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	34m 33s		trunk passed
+1 💚	compile	0m 41s		trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	compile	0m 37s		trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	checkstyle	0m 27s		trunk passed
+1 💚	mvnsite	0m 42s		trunk passed
+1 💚	javadoc	0m 42s		trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	0m 56s		trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	spotbugs	1m 18s		trunk passed
+1 💚	shadedclient	14m 31s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 35s		the patch passed
+1 💚	compile	0m 33s		the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javac	0m 33s		the patch passed
+1 💚	compile	0m 30s		the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	javac	0m 30s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 18s		the patch passed
+1 💚	mvnsite	0m 32s		the patch passed
+1 💚	javadoc	0m 30s		the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	0m 49s		the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	spotbugs	1m 16s		the patch passed
+1 💚	shadedclient	14m 13s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	17m 52s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt	hadoop-hdfs-rbf in the patch passed.
+1 💚	asflicense	0m 37s		The patch does not generate ASF License warnings.
		94m 26s

Reason	Tests
Failed junit tests	hadoop.hdfs.server.federation.router.TestRouterWebHdfsMethods

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2954/2/artifact/out/Dockerfile
GITHUB PR	#2954
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 16a1455df276 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `c8b3b27`
Default Java	Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2954/2/testReport/
Max. process+thread count	2920 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2954/2/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

Hexiaoqiao · 2021-04-27T08:45:55Z

@fengnanli unit test TestRouterWebHdfsMethods run failed. Would you mind to check it or trigger Yetus again?

fengnanli · 2021-04-27T16:56:36Z

@fengnanli unit test TestRouterWebHdfsMethods run failed. Would you mind to check it or trigger Yetus again?

Yes, I checked this test locally and it works fine. Even for the first commit it succeeded.

fengnanli · 2021-04-27T17:57:36Z

@fengnanli unit test TestRouterWebHdfsMethods run failed. Would you mind to check it or trigger Yetus again?

Yes, I checked this test locally and it works fine. Even for the first commit it succeeded.

I further checked and found this sometimes doesn't work in local env as well. It looks like a flaky test but irrelevant to this path. I am working on fixing it.

fengnanli · 2021-04-29T05:37:09Z

@Hexiaoqiao I fixed the tests and it is not related with this jira though. Let me know if want me to create a separate task or just piggyback here since it is a one-liner fix.

hadoop-yetus · 2021-04-29T07:11:31Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 39s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	34m 29s		trunk passed
+1 💚	compile	0m 43s		trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	compile	0m 37s		trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	checkstyle	0m 26s		trunk passed
+1 💚	mvnsite	0m 43s		trunk passed
+1 💚	javadoc	0m 41s		trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	0m 53s		trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	spotbugs	1m 19s		trunk passed
+1 💚	shadedclient	14m 16s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 33s		the patch passed
+1 💚	compile	0m 33s		the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javac	0m 33s		the patch passed
+1 💚	compile	0m 29s		the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	javac	0m 29s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 18s		the patch passed
+1 💚	mvnsite	0m 31s		the patch passed
+1 💚	javadoc	0m 31s		the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	0m 47s		the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	spotbugs	1m 16s		the patch passed
+1 💚	shadedclient	14m 17s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	18m 31s		hadoop-hdfs-rbf in the patch passed.
+1 💚	asflicense	0m 33s		The patch does not generate ASF License warnings.
		94m 37s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2954/4/artifact/out/Dockerfile
GITHUB PR	#2954
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 25436df36b00 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `1023095`
Default Java	Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2954/4/testReport/
Max. process+thread count	2395 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2954/4/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

Hexiaoqiao

Thanks @fengnanli for your works.
LGTM. +1 from my side. Will commit shortly if no other objections.

…ache#2954). Contributed by Fengnan Li. Reviewed-by: He Xiaoqiao <[email protected]>

fengnanli changed the title ~~HDFS-15561 RBF:Remove NPE when local namenode is not configured~~ HDFS-15561. RBF:Remove NPE when local namenode is not configured Apr 25, 2021

Hexiaoqiao reviewed Apr 25, 2021

View reviewed changes

goiri changed the title ~~HDFS-15561. RBF:Remove NPE when local namenode is not configured~~ HDFS-15561. RBF: Remove NPE when local namenode is not configured Apr 25, 2021

HDFS-15561 RBF:Remove NPE when local namenode is not configured

c8b3b27

fengnanli force-pushed the HDFS-15561 branch from 4d83dc4 to c8b3b27 Compare April 26, 2021 18:10

Fix tests

1023095

Hexiaoqiao approved these changes Apr 29, 2021

View reviewed changes

Hexiaoqiao merged commit 552e9dc into apache:trunk Apr 29, 2021

aajisaka mentioned this pull request May 2, 2021

HADOOP-17653. Do not use guava's Files.createTempDir(). #2945

Merged

kiran-maturi pushed a commit to kiran-maturi/hadoop that referenced this pull request Nov 24, 2021

HDFS-15561. RBF: Remove NPE when local namenode is not configured (ap…

41483c3

…ache#2954). Contributed by Fengnan Li. Reviewed-by: He Xiaoqiao <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

HDFS-15561. RBF: Remove NPE when local namenode is not configured #2954

HDFS-15561. RBF: Remove NPE when local namenode is not configured #2954

Uh oh!

fengnanli commented Apr 25, 2021

Uh oh!

hadoop-yetus commented Apr 25, 2021

Uh oh!

Hexiaoqiao left a comment

Uh oh!

fengnanli commented Apr 26, 2021

Uh oh!

hadoop-yetus commented Apr 26, 2021

Uh oh!

Hexiaoqiao commented Apr 27, 2021

Uh oh!

fengnanli commented Apr 27, 2021

Uh oh!

fengnanli commented Apr 27, 2021

Uh oh!

fengnanli commented Apr 29, 2021

Uh oh!

hadoop-yetus commented Apr 29, 2021

Uh oh!

Hexiaoqiao left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

HDFS-15561. RBF: Remove NPE when local namenode is not configured #2954

HDFS-15561. RBF: Remove NPE when local namenode is not configured #2954

Uh oh!

Conversation

fengnanli commented Apr 25, 2021

NOTICE

Uh oh!

hadoop-yetus commented Apr 25, 2021

Uh oh!

Hexiaoqiao left a comment

Choose a reason for hiding this comment

Uh oh!

fengnanli commented Apr 26, 2021

Uh oh!

hadoop-yetus commented Apr 26, 2021

Uh oh!

Hexiaoqiao commented Apr 27, 2021

Uh oh!

fengnanli commented Apr 27, 2021

Uh oh!

fengnanli commented Apr 27, 2021

Uh oh!

fengnanli commented Apr 29, 2021

Uh oh!

hadoop-yetus commented Apr 29, 2021

Uh oh!

Hexiaoqiao left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants