HADOOP-18443. Upgrade snakeyaml to 1.32 #4873

hotcodemacha · 2022-09-08T19:35:15Z

Description of PR

Upgrade snakeyaml to 1.32 to mitigate CVE-2022-25857 and and CVE-2022-38752 for branch-3.3.

Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

hadoop-yetus · 2022-09-09T11:39:15Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	10m 48s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+0 🆗	shelldocs	0m 1s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ branch-3.3 Compile Tests _
+0 🆗	mvndep	14m 56s		Maven dependency ordering for branch
+1 💚	mvninstall	23m 55s		branch-3.3 passed
+1 💚	compile	18m 15s		branch-3.3 passed
+1 💚	mvnsite	20m 40s		branch-3.3 passed
+1 💚	javadoc	7m 13s		branch-3.3 passed
+1 💚	shadedclient	30m 18s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 35s		Maven dependency ordering for patch
+1 💚	mvninstall	22m 6s		the patch passed
+1 💚	compile	17m 40s		the patch passed
+1 💚	javac	17m 40s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	20m 18s		the patch passed
+1 💚	shellcheck	0m 0s		No new issues.
+1 💚	javadoc	6m 57s		the patch passed
+1 💚	shadedclient	31m 20s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	746m 56s	/patch-unit-root.txt	root in the patch passed.
+1 💚	asflicense	2m 19s		The patch does not generate ASF License warnings.
		962m 59s

Reason	Tests
Failed junit tests	hadoop.hdfs.server.namenode.TestNameNodeMXBean
	hadoop.hdfs.server.namenode.TestFileTruncate
	hadoop.hdfs.server.namenode.ha.TestHAAppend
	hadoop.yarn.client.api.impl.TestAMRMClient
	hadoop.yarn.sls.appmaster.TestAMSimulator

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4873/1/artifact/out/Dockerfile
GITHUB PR	#4873
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint shellcheck shelldocs
uname	Linux 27cc55b7a77e 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	branch-3.3 / `0087981`
Default Java	Private Build-1.8.0_342-8u342-b07-0ubuntu1~18.04-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4873/1/testReport/
Max. process+thread count	3276 (vs. ulimit of 5500)
modules	C: hadoop-project . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4873/1/console
versions	git=2.17.1 maven=3.6.0 shellcheck=0.4.6
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hotcodemacha · 2022-09-09T13:12:55Z

Failures look unrelated and passing in local.
hadoop.hdfs.server.namenode.TestNameNodeMXBean -> Verified and passing in local
hadoop.hdfs.server.namenode.TestFileTruncate -> Verified and passing in local
hadoop.hdfs.server.namenode.ha.TestHAAppend -> Verified and passing in local
hadoop.yarn.client.api.impl.TestAMRMClient -> Verified and passing in local
hadoop.yarn.sls.appmaster.TestAMSimulator -> Verified and passing in local

hadoop-yetus · 2022-09-13T17:55:17Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 50s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+0 🆗	xmllint	0m 1s		xmllint was not available.
+0 🆗	shelldocs	0m 1s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ branch-3.3 Compile Tests _
+0 🆗	mvndep	15m 28s		Maven dependency ordering for branch
+1 💚	mvninstall	24m 13s		branch-3.3 passed
+1 💚	compile	18m 5s		branch-3.3 passed
+1 💚	mvnsite	20m 36s		branch-3.3 passed
+1 💚	javadoc	7m 14s		branch-3.3 passed
+1 💚	shadedclient	30m 30s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 34s		Maven dependency ordering for patch
+1 💚	mvninstall	21m 57s		the patch passed
+1 💚	compile	17m 37s		the patch passed
+1 💚	javac	17m 37s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	20m 15s		the patch passed
+1 💚	shellcheck	0m 0s		No new issues.
+1 💚	javadoc	6m 58s		the patch passed
+1 💚	shadedclient	31m 16s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	744m 57s	/patch-unit-root.txt	root in the patch passed.
+1 💚	asflicense	2m 10s		The patch does not generate ASF License warnings.
		951m 38s

Reason	Tests
Failed junit tests	hadoop.hdfs.TestDecommissionWithStriped
	hadoop.hdfs.TestReconstructStripedFile
	hadoop.hdfs.TestViewDistributedFileSystem
	hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4873/2/artifact/out/Dockerfile
GITHUB PR	#4873
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint shellcheck shelldocs
uname	Linux c157e0358827 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	branch-3.3 / `044fd6f`
Default Java	Private Build-1.8.0_342-8u342-b07-0ubuntu1~18.04-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4873/2/testReport/
Max. process+thread count	3066 (vs. ulimit of 5500)
modules	C: hadoop-project . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4873/2/console
versions	git=2.17.1 maven=3.6.0 shellcheck=0.4.6
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hotcodemacha · 2022-09-13T19:17:44Z

UT Failures in 2nd last build are unrelated to this change.

Following UTs are passing while running in local.

hadoop.hdfs.TestDecommissionWithStriped
hadoop.hdfs.TestReconstructStripedFile
hadoop.hdfs.TestViewDistributedFileSystem

hadoop-yetus · 2022-09-19T16:52:17Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	6m 58s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+0 🆗	shelldocs	0m 0s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ branch-3.3 Compile Tests _
+0 🆗	mvndep	15m 31s		Maven dependency ordering for branch
+1 💚	mvninstall	26m 14s		branch-3.3 passed
+1 💚	compile	19m 53s		branch-3.3 passed
+1 💚	mvnsite	22m 30s		branch-3.3 passed
+1 💚	javadoc	7m 43s		branch-3.3 passed
+1 💚	shadedclient	31m 45s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 37s		Maven dependency ordering for patch
+1 💚	mvninstall	24m 27s		the patch passed
+1 💚	compile	19m 21s		the patch passed
+1 💚	javac	19m 21s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	22m 20s		the patch passed
+1 💚	shellcheck	0m 0s		No new issues.
+1 💚	javadoc	7m 29s		the patch passed
+1 💚	shadedclient	32m 39s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	709m 21s	/patch-unit-root.txt	root in the patch passed.
+1 💚	asflicense	2m 23s		The patch does not generate ASF License warnings.
		936m 47s

Reason	Tests
Failed junit tests	hadoop.fs.contract.router.web.TestRouterWebHDFSContractCreate
	hadoop.hdfs.server.datanode.TestDataNodeRollingUpgrade
	hadoop.hdfs.server.datanode.TestDataNodeErasureCodingMetrics
	hadoop.yarn.sls.appmaster.TestAMSimulator

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4873/4/artifact/out/Dockerfile
GITHUB PR	#4873
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint shellcheck shelldocs
uname	Linux 75edc11803a1 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	branch-3.3 / `5d89d88`
Default Java	Private Build-1.8.0_342-8u342-b07-0ubuntu1~18.04-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4873/4/testReport/
Max. process+thread count	3156 (vs. ulimit of 5500)
modules	C: hadoop-project . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4873/4/console
versions	git=2.17.1 maven=3.6.0 shellcheck=0.4.6
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2022-09-19T17:08:50Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	10m 23s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+0 🆗	shelldocs	0m 1s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ branch-3.3 Compile Tests _
+0 🆗	mvndep	14m 52s		Maven dependency ordering for branch
+1 💚	mvninstall	27m 20s		branch-3.3 passed
+1 💚	compile	18m 59s		branch-3.3 passed
+1 💚	mvnsite	21m 26s		branch-3.3 passed
+1 💚	javadoc	7m 9s		branch-3.3 passed
+1 💚	shadedclient	32m 0s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 29s		Maven dependency ordering for patch
+1 💚	mvninstall	24m 59s		the patch passed
+1 💚	compile	18m 26s		the patch passed
+1 💚	javac	18m 26s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	20m 54s		the patch passed
+1 💚	shellcheck	0m 0s		No new issues.
+1 💚	javadoc	7m 4s		the patch passed
+1 💚	shadedclient	32m 50s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	727m 29s	/patch-unit-root.txt	root in the patch passed.
+1 💚	asflicense	2m 11s		The patch does not generate ASF License warnings.
		955m 4s

Reason	Tests
Failed junit tests	hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped
	hadoop.hdfs.TestRollingUpgrade

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4873/3/artifact/out/Dockerfile
GITHUB PR	#4873
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint shellcheck shelldocs
uname	Linux bf0c86e8f29c 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	branch-3.3 / `5d89d88`
Default Java	Private Build-1.8.0_342-8u342-b07-0ubuntu1~18.04-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4873/3/testReport/
Max. process+thread count	2304 (vs. ulimit of 5500)
modules	C: hadoop-project . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4873/3/console
versions	git=2.17.1 maven=3.6.0 shellcheck=0.4.6
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2022-09-20T19:00:04Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 54s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+0 🆗	xmllint	0m 1s		xmllint was not available.
+0 🆗	shelldocs	0m 1s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ branch-3.3 Compile Tests _
+0 🆗	mvndep	14m 39s		Maven dependency ordering for branch
+1 💚	mvninstall	27m 19s		branch-3.3 passed
+1 💚	compile	19m 9s		branch-3.3 passed
+1 💚	mvnsite	21m 32s		branch-3.3 passed
+1 💚	javadoc	7m 6s		branch-3.3 passed
+1 💚	shadedclient	32m 9s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 29s		Maven dependency ordering for patch
+1 💚	mvninstall	25m 21s		the patch passed
+1 💚	compile	18m 26s		the patch passed
+1 💚	javac	18m 26s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	21m 15s		the patch passed
+1 💚	shellcheck	0m 0s		No new issues.
+1 💚	javadoc	7m 1s		the patch passed
+1 💚	shadedclient	32m 54s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	729m 2s	/patch-unit-root.txt	root in the patch passed.
+1 💚	asflicense	2m 35s		The patch does not generate ASF License warnings.
		948m 29s

Reason	Tests
Failed junit tests	hadoop.hdfs.server.balancer.TestBalancerRPCDelay
	hadoop.hdfs.TestRollingUpgrade
	hadoop.hdfs.server.balancer.TestBalancer

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4873/5/artifact/out/Dockerfile
GITHUB PR	#4873
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint shellcheck shelldocs
uname	Linux ebdc7925b0b1 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	branch-3.3 / `339f10a`
Default Java	Private Build-1.8.0_342-8u342-b07-0ubuntu1~18.04-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4873/5/testReport/
Max. process+thread count	2435 (vs. ulimit of 5500)
modules	C: hadoop-project . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4873/5/console
versions	git=2.17.1 maven=3.6.0 shellcheck=0.4.6
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hotcodemacha · 2022-09-23T21:18:56Z

UT failures are not related.

aajisaka · 2022-09-25T14:47:55Z

UT failures are related.

Probably you meant they are not related. I ran the tests locally and they passed.

hotcodemacha · 2022-09-25T14:53:20Z

UT failures are related.

Probably you meant they are not related. I ran the tests locally and they passed.

Thanks @aajisaka for correcting. I meant they are not related. (Edited my previous comment as well)

HADOOP-18443. Upgrade snakeyaml to 1.31 to mitigate CVE-2022-25857

0087981

Trigger jenkins

7f3f161

hotcodemacha force-pushed the HADOOP-18443-branch-3.3 branch from 044fd6f to 7f3f161 Compare September 19, 2022 01:12

jenkins trigger

5d89d88

hotcodemacha changed the title ~~HADOOP-18443. Upgrade snakeyaml to 1.31 to mitigate CVE-2022-25857~~ Upgrade snakeyaml to 1.32 to mitigate CVE-2022-25857 and and CVE-2022-38752 Sep 19, 2022

hotcodemacha changed the title ~~Upgrade snakeyaml to 1.32 to mitigate CVE-2022-25857 and and CVE-2022-38752~~ Upgrade snakeyaml to 1.32 Sep 19, 2022

Trigger Build

339f10a

hotcodemacha mentioned this pull request Sep 23, 2022

HADOOP-18443. Upgrade snakeyaml to 1.32 #4906

Merged

4 tasks

aajisaka changed the title ~~Upgrade snakeyaml to 1.32~~ HADOOP-18443. Upgrade snakeyaml to 1.32 in branch-3.3 Sep 25, 2022

aajisaka changed the title ~~HADOOP-18443. Upgrade snakeyaml to 1.32 in branch-3.3~~ HADOOP-18443. Upgrade snakeyaml to 1.32 Sep 25, 2022

aajisaka approved these changes Sep 25, 2022

View reviewed changes

aajisaka merged commit 51605f9 into apache:branch-3.3 Sep 25, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

HADOOP-18443. Upgrade snakeyaml to 1.32 #4873

HADOOP-18443. Upgrade snakeyaml to 1.32 #4873

Uh oh!

hotcodemacha commented Sep 8, 2022 •

edited by aajisaka

Loading

Uh oh!

hadoop-yetus commented Sep 9, 2022

Uh oh!

hotcodemacha commented Sep 9, 2022

Uh oh!

hadoop-yetus commented Sep 13, 2022

Uh oh!

hotcodemacha commented Sep 13, 2022 •

edited

Loading

Uh oh!

hadoop-yetus commented Sep 19, 2022

Uh oh!

hadoop-yetus commented Sep 19, 2022

Uh oh!

hadoop-yetus commented Sep 20, 2022

Uh oh!

hotcodemacha commented Sep 23, 2022 •

edited

Loading

Uh oh!

aajisaka commented Sep 25, 2022

Uh oh!

hotcodemacha commented Sep 25, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

HADOOP-18443. Upgrade snakeyaml to 1.32 #4873

HADOOP-18443. Upgrade snakeyaml to 1.32 #4873

Uh oh!

Conversation

hotcodemacha commented Sep 8, 2022 • edited by aajisaka Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of PR

Uh oh!

hadoop-yetus commented Sep 9, 2022

Uh oh!

hotcodemacha commented Sep 9, 2022

Uh oh!

hadoop-yetus commented Sep 13, 2022

Uh oh!

hotcodemacha commented Sep 13, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hadoop-yetus commented Sep 19, 2022

Uh oh!

hadoop-yetus commented Sep 19, 2022

Uh oh!

hadoop-yetus commented Sep 20, 2022

Uh oh!

hotcodemacha commented Sep 23, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aajisaka commented Sep 25, 2022

Uh oh!

hotcodemacha commented Sep 25, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

hotcodemacha commented Sep 8, 2022 •

edited by aajisaka

Loading

hotcodemacha commented Sep 13, 2022 •

edited

Loading

hotcodemacha commented Sep 23, 2022 •

edited

Loading