HDFS-16402. Improve HeartbeatManager logic to avoid incorrect stats. #3839

tomscut · 2021-12-29T00:23:11Z

After reconfig dfs.datanode.data.dir, we found that the stats of the Namenode Web became negative and there were many NPE in namenode logs. This problem has been solved by HDFS-14042.

However, if HeartbeatManager#updateHeartbeat and HeartbeatManager#updateLifeline throw other exceptions, stats errors can also occur. We should ensure that stats.subtract() and stats.add() are transactional.

hadoop-yetus · 2021-12-29T07:40:21Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 49s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	35m 44s		trunk passed
+1 💚	compile	1m 28s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	1m 19s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	0m 59s		trunk passed
+1 💚	mvnsite	1m 25s		trunk passed
+1 💚	javadoc	1m 2s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 28s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	3m 22s		trunk passed
+1 💚	shadedclient	25m 29s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 18s		the patch passed
+1 💚	compile	1m 22s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	1m 22s		the patch passed
+1 💚	compile	1m 13s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	1m 13s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 52s		the patch passed
+1 💚	mvnsite	1m 18s		the patch passed
+1 💚	javadoc	0m 52s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 23s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	3m 26s		the patch passed
+1 💚	shadedclient	26m 2s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	327m 24s		hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 37s		The patch does not generate ASF License warnings.
		435m 53s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3839/1/artifact/out/Dockerfile
GITHUB PR	#3839
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 873c2d183bb1 4.15.0-163-generic #171-Ubuntu SMP Fri Nov 5 11:55:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `aa9d313`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3839/1/testReport/
Max. process+thread count	2229 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3839/1/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

Hexiaoqiao · 2022-01-09T07:32:44Z

It's good catch here. Would you mind to add new test to cover this case?
BTW, what is the root cause about NPE here? Thanks.

tomscut · 2022-01-10T01:52:13Z

It's good catch here. Would you mind to add new test to cover this case? BTW, what is the root cause about NPE here? Thanks.

Thanks @Hexiaoqiao for your review and comments.

The main reason is that the DatanodeDescriptor#storageMap cannot find the storage that DN reports through lifeline. As a result, the NPE is thrown in line 460 of the code when the storage is operated. It is similar to the situation encountered in this issue HDFS-14042.

Here is the scenario we encountered:
1 We add a disk to datanode and reconfig dfs.datanode.data.dir.
2 After a while, many values on NN's Web become negative.
3 We found the NPE from the NN log and discovered this issue HDFS-14042.

Hexiaoqiao · 2022-01-10T04:52:40Z

Thanks @tomscut for your information. It makes sense to me. I concerned why HDFS-14042 not fix completely, any other corner case?

tomscut · 2022-01-10T06:09:59Z

Thanks @tomscut for your information. It makes sense to me. I concerned why HDFS-14042 not fix completely, any other corner case?

Thanks @Hexiaoqiao for your comments. NPE has been fixed by HDFS-14042. IMO, it is a safer way to keep these two operations (stats.subtract() and stats.add()) transactional in case of any other potential exceptions. What do you think of this? This has no effect on the original logic.

Hexiaoqiao · 2022-01-10T06:24:34Z

IMO, it is a safer way to keep these two operations (stats.subtract() and stats.add()) transactional in case of any other potential exceptions.
Totally agree with that. Not sure why/where throw NPE after HDFS-14042. I think add another new unit test to cover will be better.

tomscut · 2022-01-10T06:26:54Z

IMO, it is a safer way to keep these two operations (stats.subtract() and stats.add()) transactional in case of any other potential exceptions.
Totally agree with that. Not sure why/where throw NPE after HDFS-14042. I think add another new unit test to cover will be better.

Ok, I'll try to add a unit test later. Thanks for your review.

hadoop-yetus · 2022-01-16T14:15:49Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 57s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	35m 16s		trunk passed
+1 💚	compile	1m 28s		trunk passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	compile	1m 20s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	checkstyle	1m 0s		trunk passed
+1 💚	mvnsite	1m 26s		trunk passed
+1 💚	javadoc	1m 2s		trunk passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	1m 28s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	3m 23s		trunk passed
+1 💚	shadedclient	25m 31s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 19s		the patch passed
+1 💚	compile	1m 23s		the patch passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	javac	1m 23s		the patch passed
+1 💚	compile	1m 16s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	javac	1m 16s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 55s	/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs-project/hadoop-hdfs: The patch generated 2 new + 160 unchanged - 0 fixed = 162 total (was 160)
+1 💚	mvnsite	1m 20s		the patch passed
+1 💚	javadoc	0m 56s		the patch passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	1m 22s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	3m 31s		the patch passed
+1 💚	shadedclient	25m 29s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	326m 23s		hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 38s		The patch does not generate ASF License warnings.
		434m 27s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3839/2/artifact/out/Dockerfile
GITHUB PR	#3839
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 1cca29272d2e 4.15.0-163-generic #171-Ubuntu SMP Fri Nov 5 11:55:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `9ccf3f7`
Default Java	Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3839/2/testReport/
Max. process+thread count	2366 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3839/2/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2022-01-16T21:34:11Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 53s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	35m 33s		trunk passed
+1 💚	compile	1m 29s		trunk passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	compile	1m 18s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	checkstyle	1m 0s		trunk passed
+1 💚	mvnsite	1m 27s		trunk passed
+1 💚	javadoc	1m 2s		trunk passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	1m 29s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	3m 28s		trunk passed
+1 💚	shadedclient	25m 14s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 22s		the patch passed
+1 💚	compile	1m 23s		the patch passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	javac	1m 23s		the patch passed
+1 💚	compile	1m 13s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	javac	1m 13s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 55s		the patch passed
+1 💚	mvnsite	1m 22s		the patch passed
+1 💚	javadoc	0m 53s		the patch passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	1m 20s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	3m 24s		the patch passed
+1 💚	shadedclient	25m 35s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	310m 50s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 37s		The patch does not generate ASF License warnings.
		418m 48s

Reason	Tests
Failed junit tests	hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3839/3/artifact/out/Dockerfile
GITHUB PR	#3839
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux a21e68017757 4.15.0-163-generic #171-Ubuntu SMP Fri Nov 5 11:55:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `5e4c319`
Default Java	Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3839/3/testReport/
Max. process+thread count	2238 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3839/3/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

tomscut · 2022-01-17T23:55:47Z

Hi @Hexiaoqiao , I added an unit test for this change. Please take a look. Thanks.

tomscut · 2022-01-17T23:56:43Z

The failed unit test TestDataNodeHotSwapVolumes is unrelated to the change.

Hexiaoqiao

LGTM. +1. The failed unit test seems not to related to this changes. re-trigger jenkins and let's wait what he says.

hadoop-yetus · 2022-01-23T15:36:16Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	44m 53s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	35m 57s		trunk passed
+1 💚	compile	1m 16s		trunk passed
+1 💚	checkstyle	1m 1s		trunk passed
+1 💚	mvnsite	1m 25s		trunk passed
+1 💚	javadoc	1m 30s		trunk passed
+1 💚	spotbugs	3m 22s		trunk passed
+1 💚	shadedclient	25m 13s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 18s		the patch passed
+1 💚	compile	1m 13s		the patch passed
+1 💚	javac	1m 13s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 53s		the patch passed
+1 💚	mvnsite	1m 19s		the patch passed
+1 💚	javadoc	1m 25s		the patch passed
+1 💚	spotbugs	3m 26s		the patch passed
+1 💚	shadedclient	25m 7s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	302m 13s		hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 39s		The patch does not generate ASF License warnings.
		448m 47s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3839/4/artifact/out/Dockerfile
GITHUB PR	#3839
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 81eafc7da3c5 4.15.0-163-generic #171-Ubuntu SMP Fri Nov 5 11:55:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `5e4c319`
Default Java	Red Hat, Inc.-1.8.0_312-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3839/4/testReport/
Max. process+thread count	2335 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3839/4/console
versions	git=2.9.5 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

tomscut · 2022-01-23T15:44:58Z

LGTM. +1. The failed unit test seems not to related to this changes. re-trigger jenkins and let's wait what he says.

Thanks @Hexiaoqiao for your review.

hadoop-yetus · 2022-01-23T22:48:11Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	25m 36s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	24m 51s		trunk passed
+1 💚	compile	1m 33s		trunk passed
+1 💚	checkstyle	1m 11s		trunk passed
+1 💚	mvnsite	1m 37s		trunk passed
+1 💚	javadoc	1m 41s		trunk passed
+1 💚	spotbugs	3m 37s		trunk passed
+1 💚	shadedclient	25m 44s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 22s		the patch passed
+1 💚	compile	1m 21s		the patch passed
+1 💚	javac	1m 21s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 59s		the patch passed
+1 💚	mvnsite	1m 25s		the patch passed
+1 💚	javadoc	1m 28s		the patch passed
+1 💚	spotbugs	3m 31s		the patch passed
+1 💚	shadedclient	25m 9s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	312m 55s		hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 56s		The patch does not generate ASF License warnings.
		431m 43s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3839/4/artifact/out/Dockerfile
GITHUB PR	#3839
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 9e7444c79a7d 4.15.0-163-generic #171-Ubuntu SMP Fri Nov 5 11:55:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `5e4c319`
Default Java	Red Hat, Inc.-1.8.0_312-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3839/4/testReport/
Max. process+thread count	2053 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3839/4/console
versions	git=2.27.0 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2022-01-24T06:17:21Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	16m 18s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	32m 1s		trunk passed
+1 💚	compile	1m 28s		trunk passed
+1 💚	checkstyle	1m 5s		trunk passed
+1 💚	mvnsite	1m 33s		trunk passed
+1 💚	javadoc	0m 57s		trunk passed
-1 ❌	spotbugs	3m 41s	/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html	hadoop-hdfs-project/hadoop-hdfs in trunk has 6 extant spotbugs warnings.
+1 💚	shadedclient	34m 26s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 32s		the patch passed
+1 💚	compile	1m 22s		the patch passed
+1 💚	javac	1m 22s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 58s		the patch passed
+1 💚	mvnsite	1m 32s		the patch passed
+1 💚	javadoc	0m 53s		the patch passed
+1 💚	spotbugs	3m 51s		the patch passed
+1 💚	shadedclient	34m 20s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	316m 10s		hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 38s		The patch does not generate ASF License warnings.
		448m 58s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3839/4/artifact/out/Dockerfile
GITHUB PR	#3839
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 0495e3b2ce37 4.15.0-163-generic #171-Ubuntu SMP Fri Nov 5 11:55:11 UTC 2021 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `5e4c319`
Default Java	Debian-11.0.13+8-post-Debian-1deb10u1
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3839/4/testReport/
Max. process+thread count	2190 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3839/4/console
versions	git=2.20.1 maven=3.6.0 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

…pache#3839). Contributed by tomscut. Reviewed-by: He Xiaoqiao <[email protected]>

HDFS-16402. HeartbeatManager may cause incorrect stats

aa9d313

add unit test

9ccf3f7

fix checkstyles

5e4c319

Hexiaoqiao approved these changes Jan 23, 2022

View reviewed changes

Hexiaoqiao changed the title ~~HDFS-16402. HeartbeatManager may cause incorrect stats~~ HDFS-16402. Improve HeartbeatManager logic to avoid incorrect stats. Jan 24, 2022

Hexiaoqiao merged commit 15b820c into apache:trunk Jan 24, 2022

HarshitGupta11 pushed a commit to HarshitGupta11/hadoop that referenced this pull request Nov 28, 2022

HDFS-16402. Improve HeartbeatManager logic to avoid incorrect stats. (a…

9d7b807

…pache#3839). Contributed by tomscut. Reviewed-by: He Xiaoqiao <[email protected]>

HDFS-16402. Improve HeartbeatManager logic to avoid incorrect stats. #3839

HDFS-16402. Improve HeartbeatManager logic to avoid incorrect stats. #3839

Uh oh!

Conversation

tomscut commented Dec 29, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hadoop-yetus commented Dec 29, 2021

Uh oh!

Hexiaoqiao commented Jan 9, 2022

Uh oh!

tomscut commented Jan 10, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Hexiaoqiao commented Jan 10, 2022

Uh oh!

tomscut commented Jan 10, 2022

Uh oh!

Hexiaoqiao commented Jan 10, 2022

Uh oh!

tomscut commented Jan 10, 2022

Uh oh!

hadoop-yetus commented Jan 16, 2022

Uh oh!

hadoop-yetus commented Jan 16, 2022

Uh oh!

tomscut commented Jan 17, 2022

Uh oh!

tomscut commented Jan 17, 2022

Uh oh!

Hexiaoqiao left a comment

Choose a reason for hiding this comment

Uh oh!

hadoop-yetus commented Jan 23, 2022

Uh oh!

tomscut commented Jan 23, 2022

Uh oh!

hadoop-yetus commented Jan 23, 2022

Uh oh!

hadoop-yetus commented Jan 24, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

tomscut commented Dec 29, 2021 •

edited

Loading

tomscut commented Jan 10, 2022 •

edited

Loading