YARN-11824: Prevent test dependencies from leaking into the distro.

[cnauroth]

Description of PR

First, make sure test libraries like jersey-test-framework-provider-jetty are limited to scope test.

Then, update other modules that didn't know they were getting their test dependencies through YARN to depend on test libraries explicitly.

How was this patch tested?

I used mvn dependency:tree to check that the test dependencies were gone after these changes. Before the changes, we had this in the final distro after mvn clean package -Pdist -Dtar -DskipTests -DskipShade:

/tmp/distro/hadoop-3.5.0-SNAPSHOT/share/hadoop/mapreduce/lib/mockito-junit-jupiter-4.11.0.jar
/tmp/distro/hadoop-3.5.0-SNAPSHOT/share/hadoop/yarn/lib/junit-jupiter-5.11.0.jar
/tmp/distro/hadoop-3.5.0-SNAPSHOT/share/hadoop/tools/lib/junit-4.13.2.jar
/tmp/distro/hadoop-3.5.0-SNAPSHOT/share/hadoop/tools/lib/junit-jupiter-engine-5.13.3.jar
/tmp/distro/hadoop-3.5.0-SNAPSHOT/share/hadoop/tools/lib/junit-platform-commons-1.13.3.jar
/tmp/distro/hadoop-3.5.0-SNAPSHOT/share/hadoop/tools/lib/junit-platform-engine-1.13.3.jar
/tmp/distro/hadoop-3.5.0-SNAPSHOT/share/hadoop/tools/lib/junit-jupiter-params-5.13.3.jar
/tmp/distro/hadoop-3.5.0-SNAPSHOT/share/hadoop/tools/lib/junit-jupiter-api-5.13.3.jar
/tmp/distro/hadoop-3.5.0-SNAPSHOT/share/hadoop/yarn/lib/hamcrest-3.0.jar
/tmp/distro/hadoop-3.5.0-SNAPSHOT/share/hadoop/tools/lib/hamcrest-core-1.3.jar

After these changes, everything is gone except for the stuff in hadoop/tools/lib. This is apparently intentional dependencies of hadoop-tools/hadoop-compat-bench, but it won't creep into the classpath for dependent projects like Parquet.

For code changes:

• Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
• Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
• If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
• If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

[cnauroth]

@steveloughran and @slfan1989 , review please? Thank you!

[pan3793]

I also spotted this issue today, thank you for fixing it!

[slfan1989]

@steveloughran and @slfan1989 , review please? Thank you!

@cnauroth Many thanks for spotting and fixing this issue. There are still some follow-up issues with the Jersey upgrade that we’ll need to resolve step by step.

[hadoop-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 31s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+0 🆗	mvndep	8m 53s		Maven dependency ordering for branch
+1 💚	mvninstall	33m 7s		trunk passed
+1 💚	compile	16m 1s		trunk passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚	compile	14m 1s		trunk passed with JDK Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
+1 💚	mvnsite	4m 20s		trunk passed
+1 💚	javadoc	4m 0s		trunk passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	4m 3s		trunk passed with JDK Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
+1 💚	shadedclient	120m 18s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 34s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 9s		the patch passed
+1 💚	compile	14m 54s		the patch passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚	javac	14m 54s		the patch passed
+1 💚	compile	13m 36s		the patch passed with JDK Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
+1 💚	javac	13m 36s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	4m 18s		the patch passed
+1 💚	javadoc	4m 12s		the patch passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	4m 2s		the patch passed with JDK Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
+1 💚	shadedclient	50m 52s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	6m 7s		hadoop-yarn-common in the patch passed.
+1 💚	unit	4m 19s		hadoop-yarn-server-applicationhistoryservice in the patch passed.
+1 💚	unit	9m 15s		hadoop-mapreduce-client-app in the patch passed.
+1 💚	unit	22m 39s		hadoop-yarn-applications-distributedshell in the patch passed.
+1 💚	unit	21m 10s		hadoop-yarn-services-core in the patch passed.
+1 💚	asflicense	1m 22s		The patch does not generate ASF License warnings.
		269m 45s

Subsystem	Report/Notes
Docker	ClientAPI=1.51 ServerAPI=1.51 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7885/1/artifact/out/Dockerfile
GITHUB PR	#7885
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint
uname	Linux f3a628b010ba 5.15.0-140-generic #150-Ubuntu SMP Sat Apr 12 06:00:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / e5a8e3f
Default Java	Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7885/1/testReport/
Max. process+thread count	876 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7885/1/console
versions	git=2.25.1 maven=3.6.3
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

[steveloughran]

+1

[cnauroth]

Thanks for the reviews, @pan3793 , @slfan1989 and @steveloughran ! I committed this to trunk.