HDDS-13810. Fix Build Issue because of unused dependency

[swamirishi]

What changes were proposed in this pull request?

Maven module hdds-server-scm dependency analyse fails since it has dependencies jackson-databind, hdds-client and commons-compress which is only used in test however dependency has been declared for all the places whereas it should have been only declared for test scope.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-13810

How was this patch tested?

Built code locally which was failing earlier

[adoroszlai]

The dependencies were declared like this:

ozone/hadoop-hdds/server-scm/pom.xml

Lines 41 to 45 in cfcfcd5

	<dependency>
	<groupId>com.fasterxml.jackson.core</groupId>
	<artifactId>jackson-databind</artifactId>
	<!-- <scope>test</scope> but transitive via hdds-server-framework -->
	</dependency>

on purpose. For hdds-server-scm itself the dependency is for tests, but the same dependency is required at runtime transitively, for hdds-server-framework.

#8479, which introduced these comments, references the Maven bug that made this necessary. Actually MNG-8041 has a better explanation.

While build is successful both before and after your change, the patch causes SCM startup failure:

Exception in thread "main" java.lang.NoClassDefFoundError: com/fasterxml/jackson/databind/JsonSerializer
	at org.apache.hadoop.hdds.scm.server.StorageContainerManager.<init>(StorageContainerManager.java:398)
	at org.apache.hadoop.hdds.scm.server.StorageContainerManager.createSCM(StorageContainerManager.java:617)
	...
	at org.apache.hadoop.hdds.scm.server.StorageContainerManagerStarter.main(StorageContainerManagerStarter.java:59)

see CI results in your fork. (That's why you should open PRs only if CI run in your fork is successful.)

These dependencies are ignored for the test/compile scope mismatch:

ozone/pom.xml

Lines 2248 to 2262 in cfcfcd5

	<ignoredNonTestScopedDependencies>
	<!--
	Ignore false positive "Non-test scoped test only dependencies": these are test-only in some module, compile scope transitively.
	See:
	- https://issues.apache.org/jira/browse/MDEP-791
	- https://issues.apache.org/jira/browse/MNG-6058
	-->
	<ignoredNonTestScopedDependency>com.fasterxml.jackson.core:jackson-databind:jar</ignoredNonTestScopedDependency>
	<ignoredNonTestScopedDependency>org.apache.commons:commons-compress:jar</ignoredNonTestScopedDependency>
	<ignoredNonTestScopedDependency>org.apache.ozone:hdds-client:jar</ignoredNonTestScopedDependency>
	<ignoredNonTestScopedDependency>org.apache.ozone:ozone-interface-client:jar</ignoredNonTestScopedDependency>
	<ignoredNonTestScopedDependency>org.glassfish.jersey.core:jersey-common:jar</ignoredNonTestScopedDependency>
	<ignoredNonTestScopedDependency>org.rocksdb:rocksdbjni:jar</ignoredNonTestScopedDependency>
	<ignoredNonTestScopedDependency>org.springframework:spring-core:jar</ignoredNonTestScopedDependency>
	</ignoredNonTestScopedDependencies>

So there shouldn't be any "Build Issue because of unused dependency". Have you upgraded to Maven 4? If so, we need to understand why it does not respect these ignoredNonTestScopedDependencies.

Your patch is also incomplete, there are other modules with the same pattern:

hadoop-hdds/rocks-native/pom.xml
hadoop-ozone/recon/pom.xml
hadoop-ozone/s3gateway/pom.xml

[swamirishi]

I was on maven 3.8.6 upgrading to 3.9.11 fixed it somehow