HDFS-15731. Reduce threadCount for unit tests to reduce the memory usage #2556

aajisaka · 2020-12-17T08:46:59Z

JIRA: https://issues.apache.org/jira/browse/HDFS-15731

Reduce the thread count from 4 to 2.

threadCount	Test execution time (hadoop-hdfs)	Total Build Time
2	about 3 ~ 4 hours	389~470 min
3	about 2.5 hours	366 min
4	about 2 hours	338 min

hadoop-yetus · 2020-12-17T15:17:20Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 31s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚		0m 0s	test4tests	The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	13m 54s		Maven dependency ordering for branch
+1 💚	mvninstall	21m 53s		trunk passed
+1 💚	compile	20m 0s		trunk passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚	compile	17m 19s		trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
+1 💚	checkstyle	2m 39s		trunk passed
+1 💚	mvnsite	3m 2s		trunk passed
+1 💚	shadedclient	21m 32s		branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	2m 34s		trunk passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚	javadoc	3m 21s		trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
+0 🆗	spotbugs	1m 27s		Used deprecated FindBugs config; considering switching to SpotBugs.
+0 🆗	findbugs	0m 38s		branch/hadoop-project no findbugs output file (findbugsXml.xml)
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 25s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 56s		the patch passed
+1 💚	compile	19m 25s		the patch passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚	javac	19m 25s		the patch passed
+1 💚	compile	17m 15s		the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
+1 💚	javac	17m 15s		the patch passed
+1 💚	checkstyle	2m 36s		the patch passed
+1 💚	mvnsite	3m 1s		the patch passed
+1 💚	whitespace	0m 0s		The patch has no whitespace issues.
+1 💚	xml	0m 1s		The patch has no ill-formed XML file.
+1 💚	shadedclient	15m 27s		patch has no errors when building and testing our client artifacts.
+1 💚	javadoc	2m 31s		the patch passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚	javadoc	3m 18s		the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
+0 🆗	findbugs	0m 36s		hadoop-project has no data from findbugs
			_ Other Tests _
+1 💚	unit	0m 34s		hadoop-project in the patch passed.
+1 💚	unit	186m 16s		hadoop-hdfs in the patch passed.
+1 💚	unit	16m 52s		hadoop-hdfs-rbf in the patch passed.
+1 💚	asflicense	1m 8s		The patch does not generate ASF License warnings.
		389m 11s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2556/1/artifact/out/Dockerfile
GITHUB PR	#2556
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml
uname	Linux fd1cd8fc4e93 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `4c033ba`
Default Java	Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2556/1/testReport/
Max. process+thread count	3352 (vs. ulimit of 5500)
modules	C: hadoop-project hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-rbf U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2556/1/console
versions	git=2.17.1 maven=3.6.0 findbugs=4.0.6
Powered by	Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2020-12-17T16:40:43Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	1m 37s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚		0m 0s	test4tests	The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	4m 5s		Maven dependency ordering for branch
-1 ❌	mvninstall	36m 47s	/branch-mvninstall-root.txt	root in trunk failed.
+1 💚	compile	27m 37s		trunk passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚	compile	23m 11s		trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
+1 💚	checkstyle	3m 11s		trunk passed
+1 💚	mvnsite	3m 7s		trunk passed
+1 💚	shadedclient	22m 46s		branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	2m 27s		trunk passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚	javadoc	2m 58s		trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
+0 🆗	spotbugs	1m 22s		Used deprecated FindBugs config; considering switching to SpotBugs.
+0 🆗	findbugs	0m 32s		branch/hadoop-project no findbugs output file (findbugsXml.xml)
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 26s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 59s		the patch passed
+1 💚	compile	21m 41s		the patch passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚	javac	21m 41s		the patch passed
+1 💚	compile	22m 31s		the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
+1 💚	javac	22m 31s		the patch passed
+1 💚	checkstyle	3m 13s		the patch passed
+1 💚	mvnsite	3m 3s		the patch passed
+1 💚	whitespace	0m 0s		The patch has no whitespace issues.
+1 💚	xml	0m 2s		The patch has no ill-formed XML file.
+1 💚	shadedclient	19m 58s		patch has no errors when building and testing our client artifacts.
+1 💚	javadoc	2m 38s		the patch passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚	javadoc	3m 40s		the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
+0 🆗	findbugs	0m 32s		hadoop-project has no data from findbugs
			_ Other Tests _
+1 💚	unit	0m 28s		hadoop-project in the patch passed.
-1 ❌	unit	232m 39s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	unit	17m 46s		hadoop-hdfs-rbf in the patch passed.
+1 💚	asflicense	0m 54s		The patch does not generate ASF License warnings.
		470m 59s

Reason	Tests
Failed junit tests	hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped
	hadoop.hdfs.server.namenode.TestEditLogRace

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2556/2/artifact/out/Dockerfile
GITHUB PR	#2556
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml
uname	Linux 8face924dfa3 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `4c033ba`
Default Java	Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2556/2/testReport/
Max. process+thread count	2735 (vs. ulimit of 5500)
modules	C: hadoop-project hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-rbf U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2556/2/console
versions	git=2.17.1 maven=3.6.0 findbugs=4.0.6
Powered by	Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

goiri · 2020-12-17T17:21:39Z

This looks surprisingly successful.
In addition the runtime (8 hours) doesn't look that bad; for comparison how many hours was it right before?
I see some with 7 hours and some.

ayushtkn

Thanx @aajisaka for the changes.
The results do seems quite promising, We should run a couple of times and observe too.
We might need to increase the Jenkins timeouts.

In general, would be good to know what was the previous time. I think only RBF build was ~1 hr, if with HDFS, it used to ~3.5 hrs, Not sure though, Can try increasing to 4 again here itself and retry.

aajisaka · 2021-01-20T07:32:44Z

Thank you @goiri and @ayushtkn
Set the threadCount=4 to check the original execution time.

hadoop-yetus · 2021-01-20T13:11:55Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	1m 17s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚		0m 0s	test4tests	The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	13m 43s		Maven dependency ordering for branch
+1 💚	mvninstall	23m 55s		trunk passed
+1 💚	compile	21m 48s		trunk passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚	compile	18m 24s		trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
+1 💚	checkstyle	2m 53s		trunk passed
+1 💚	mvnsite	2m 40s		trunk passed
+1 💚	shadedclient	23m 7s		branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	2m 16s		trunk passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚	javadoc	3m 1s		trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
+0 🆗	spotbugs	1m 21s		Used deprecated FindBugs config; considering switching to SpotBugs.
+0 🆗	findbugs	0m 31s		branch/hadoop-project no findbugs output file (findbugsXml.xml)
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 22s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 56s		the patch passed
+1 💚	compile	21m 8s		the patch passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚	javac	21m 8s		the patch passed
+1 💚	compile	18m 22s		the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
+1 💚	javac	18m 22s		the patch passed
+1 💚	checkstyle	2m 46s		the patch passed
+1 💚	mvnsite	2m 39s		the patch passed
+1 💚	whitespace	0m 0s		The patch has no whitespace issues.
+1 💚	xml	0m 1s		The patch has no ill-formed XML file.
+1 💚	shadedclient	17m 10s		patch has no errors when building and testing our client artifacts.
+1 💚	javadoc	2m 13s		the patch passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚	javadoc	2m 59s		the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
+0 🆗	findbugs	0m 27s		hadoop-project has no data from findbugs
			_ Other Tests _
+1 💚	unit	0m 26s		hadoop-project in the patch passed.
-1 ❌	unit	131m 12s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	unit	12m 25s		hadoop-hdfs-rbf in the patch passed.
+1 💚	asflicense	1m 0s		The patch does not generate ASF License warnings.
		338m 30s

Reason	Tests
Failed junit tests	hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
	hadoop.hdfs.server.namenode.TestFsck

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2556/3/artifact/out/Dockerfile
GITHUB PR	#2556
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml
uname	Linux 364e3ba7aa54 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `4b5bc05`
Default Java	Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2556/3/testReport/
Max. process+thread count	3040 (vs. ulimit of 5500)
modules	C: hadoop-project hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-rbf U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2556/3/console
versions	git=2.17.1 maven=3.6.0 findbugs=4.0.6
Powered by	Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2021-01-21T07:09:19Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	1m 20s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚		0m 0s	test4tests	The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	13m 37s		Maven dependency ordering for branch
+1 💚	mvninstall	23m 56s		trunk passed
+1 💚	compile	21m 37s		trunk passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚	compile	18m 28s		trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
+1 💚	checkstyle	2m 48s		trunk passed
+1 💚	mvnsite	2m 38s		trunk passed
+1 💚	shadedclient	22m 57s		branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	2m 13s		trunk passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚	javadoc	2m 59s		trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
+0 🆗	spotbugs	1m 19s		Used deprecated FindBugs config; considering switching to SpotBugs.
+0 🆗	findbugs	0m 29s		branch/hadoop-project no findbugs output file (findbugsXml.xml)
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 22s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 57s		the patch passed
+1 💚	compile	21m 2s		the patch passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚	javac	21m 2s		the patch passed
+1 💚	compile	18m 19s		the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
+1 💚	javac	18m 19s		the patch passed
+1 💚	checkstyle	2m 49s		the patch passed
+1 💚	mvnsite	2m 40s		the patch passed
+1 💚	whitespace	0m 0s		The patch has no whitespace issues.
+1 💚	xml	0m 1s		The patch has no ill-formed XML file.
+1 💚	shadedclient	17m 18s		patch has no errors when building and testing our client artifacts.
+1 💚	javadoc	2m 10s		the patch passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚	javadoc	2m 59s		the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
+0 🆗	findbugs	0m 28s		hadoop-project has no data from findbugs
			_ Other Tests _
+1 💚	unit	0m 25s		hadoop-project in the patch passed.
-1 ❌	unit	159m 16s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	unit	13m 5s		hadoop-hdfs-rbf in the patch passed.
+1 💚	asflicense	0m 56s		The patch does not generate ASF License warnings.
		366m 29s

Reason	Tests
Failed junit tests	hadoop.hdfs.server.namenode.TestReconstructStripedBlocks

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2556/4/artifact/out/Dockerfile
GITHUB PR	#2556
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml
uname	Linux a0d990bbba81 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `cfe6e1f`
Default Java	Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2556/4/testReport/
Max. process+thread count	2959 (vs. ulimit of 5500)
modules	C: hadoop-project hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-rbf U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2556/4/console
versions	git=2.17.1 maven=3.6.0 findbugs=4.0.6
Powered by	Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

ayushtkn · 2021-01-21T20:34:20Z

If I am catching it correct the difference is about 2-2.5 hours, the max I see is 470 and with 4 threads it is around 338.
If it is gonna help us with OOM error, then it looks like a fair deal to me.

aajisaka · 2021-01-22T02:29:03Z

Thank you @ayushtkn. I added the elapsed time to the description.

ayushtkn · 2021-01-22T09:02:00Z

Thanx @aajisaka
I am +1 on reducing the thread count to 2. If no objections you can change the PR accordingly

This reverts commit 8eb0409.

This reverts commit 2104307.

aajisaka · 2021-01-22T09:35:58Z

Updated the PR to reduce the thread count to 2. Thank you @ayushtkn

hadoop-yetus · 2021-01-22T17:24:59Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	35m 41s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚		0m 0s	test4tests	The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	13m 39s		Maven dependency ordering for branch
+1 💚	mvninstall	23m 43s		trunk passed
+1 💚	compile	22m 1s		trunk passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚	compile	18m 26s		trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
+1 💚	checkstyle	2m 51s		trunk passed
+1 💚	mvnsite	2m 42s		trunk passed
+1 💚	shadedclient	23m 24s		branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	2m 15s		trunk passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚	javadoc	3m 1s		trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
+0 🆗	spotbugs	1m 22s		Used deprecated FindBugs config; considering switching to SpotBugs.
+0 🆗	findbugs	0m 30s		branch/hadoop-project no findbugs output file (findbugsXml.xml)
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 25s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 58s		the patch passed
+1 💚	compile	20m 59s		the patch passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚	javac	20m 59s		the patch passed
+1 💚	compile	18m 22s		the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
+1 💚	javac	18m 22s		the patch passed
+1 💚	checkstyle	2m 51s		the patch passed
+1 💚	mvnsite	2m 41s		the patch passed
+1 💚	whitespace	0m 0s		The patch has no whitespace issues.
+1 💚	xml	0m 1s		The patch has no ill-formed XML file.
+1 💚	shadedclient	17m 12s		patch has no errors when building and testing our client artifacts.
+1 💚	javadoc	2m 12s		the patch passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚	javadoc	2m 58s		the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
+0 🆗	findbugs	0m 27s		hadoop-project has no data from findbugs
			_ Other Tests _
+1 💚	unit	0m 26s		hadoop-project in the patch passed.
+1 💚	unit	221m 15s		hadoop-hdfs in the patch passed.
+1 💚	unit	17m 43s		hadoop-hdfs-rbf in the patch passed.
+1 💚	asflicense	0m 59s		The patch does not generate ASF License warnings.
		468m 29s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2556/5/artifact/out/Dockerfile
GITHUB PR	#2556
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml
uname	Linux 7db1d11024f3 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `d09e3c9`
Default Java	Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2556/5/testReport/
Max. process+thread count	2097 (vs. ulimit of 5500)
modules	C: hadoop-project hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-rbf U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2556/5/console
versions	git=2.17.1 maven=3.6.0 findbugs=4.0.6
Powered by	Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

ayushtkn

Thanx @aajisaka
Changes LGTM.

Do revert the blank line changes before committing. :-)

This reverts commit a699cf9.

aajisaka · 2021-01-24T13:50:18Z

Thank you @ayushtkn for your review. Merged.

hadoop-yetus · 2021-01-24T13:50:36Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 0s		Docker mode activated.
-1 ❌	patch	0m 5s		#2556 does not apply to trunk. Rebase required? Wrong Branch? See https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute for help.

Subsystem	Report/Notes
GITHUB PR	#2556
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2556/6/console
versions	git=2.17.1
Powered by	Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

…age (#2556) Reviewed-by: Ayush Saxena <[email protected]> (cherry picked from commit 9cf408d)

aajisaka added 2 commits December 17, 2020 17:43

HDFS-15731. Reduce threadCount for unit tests to reduce the memory usage

ca44817

Touched hadoop-hdfs and hadoop-hdfs-rbf to run the unit tests

a699cf9

tasanuma mentioned this pull request Dec 18, 2020

HADOOP-17224. Install Intel ISA-L library in Dockerfile. #2537

Merged

ayushtkn reviewed Dec 18, 2020

View reviewed changes

Use threadCount=4 to check the execution time.

2104307

Try threadsCount=3

8eb0409

aajisaka added 2 commits January 22, 2021 18:34

Revert "Try threadsCount=3"

d3077ca

This reverts commit 8eb0409.

Revert "Use threadCount=4 to check the execution time."

45d3c6d

This reverts commit 2104307.

ayushtkn approved these changes Jan 23, 2021

View reviewed changes

Revert "Touched hadoop-hdfs and hadoop-hdfs-rbf to run the unit tests"

82f3314

This reverts commit a699cf9.

aajisaka merged commit 9cf408d into apache:trunk Jan 24, 2021

aajisaka deleted the HDFS-15731 branch January 24, 2021 13:50

aajisaka added a commit that referenced this pull request Jan 24, 2021

HDFS-15731. Reduce threadCount for unit tests to reduce the memory us…

7106a78

…age (#2556) Reviewed-by: Ayush Saxena <[email protected]> (cherry picked from commit 9cf408d)

aajisaka added a commit that referenced this pull request Jan 24, 2021

HDFS-15731. Reduce threadCount for unit tests to reduce the memory us…

a5647b9

…age (#2556) Reviewed-by: Ayush Saxena <[email protected]> (cherry picked from commit 9cf408d)

aajisaka added a commit that referenced this pull request Jan 24, 2021

HDFS-15731. Reduce threadCount for unit tests to reduce the memory us…

0eee9bd

…age (#2556) Reviewed-by: Ayush Saxena <[email protected]> (cherry picked from commit 9cf408d)

HDFS-15731. Reduce threadCount for unit tests to reduce the memory usage #2556

HDFS-15731. Reduce threadCount for unit tests to reduce the memory usage #2556

Uh oh!

Conversation

aajisaka commented Dec 17, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hadoop-yetus commented Dec 17, 2020

Uh oh!

hadoop-yetus commented Dec 17, 2020

Uh oh!

goiri commented Dec 17, 2020

Uh oh!

ayushtkn left a comment

Choose a reason for hiding this comment

Uh oh!

aajisaka commented Jan 20, 2021

Uh oh!

hadoop-yetus commented Jan 20, 2021

Uh oh!

hadoop-yetus commented Jan 21, 2021

Uh oh!

ayushtkn commented Jan 21, 2021

Uh oh!

aajisaka commented Jan 22, 2021

Uh oh!

ayushtkn commented Jan 22, 2021

Uh oh!

aajisaka commented Jan 22, 2021

Uh oh!

hadoop-yetus commented Jan 22, 2021

Uh oh!

ayushtkn left a comment

Choose a reason for hiding this comment

Uh oh!

aajisaka commented Jan 24, 2021

Uh oh!

hadoop-yetus commented Jan 24, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

aajisaka commented Dec 17, 2020 •

edited

Loading