HBASE-28790 hbase-connectors fails to build with hbase 2.6.0

[NihalJain]

• Add missing method setRequestAttribute added in HBASE-27657

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 41s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+0 🆗	spotbugs	0m 0s	spotbugs executables are not available.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
-0 ⚠️	test4tests	0m 0s	The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
		_ master Compile Tests _
+1 💚	mvninstall	1m 27s	master passed
+1 💚	compile	0m 9s	master passed
+1 💚	spotless	0m 16s	branch has no errors when running spotless:check.
+1 💚	javadoc	0m 8s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	0m 51s	the patch passed
+1 💚	compile	0m 9s	the patch passed
+1 💚	javac	0m 9s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	spotless	0m 11s	patch has no errors when running spotless:check.
+1 💚	javadoc	0m 7s	the patch passed
		_ Other Tests _
+1 💚	unit	0m 10s	hbase-kafka-proxy in the patch passed.
		5m 19s

Subsystem	Report/Notes
Docker	ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hbase.apache.org/job/HBase-Connectors-PreCommit/job/PR-133/1/artifact/yetus-precommit-check/output/Dockerfile
GITHUB PR	#133
Optional Tests	dupname javac javadoc unit spotbugs findbugs spotless compile
uname	Linux 031676b5c9ba 5.4.0-186-generic #206-Ubuntu SMP Fri Apr 26 12:31:10 UTC 2024 x86_64 GNU/Linux
Build tool	hb_maven
Personality	dev-support/jenkins/hbase-personality.sh
git revision	master / 1498a91
Default Java	Oracle Corporation-1.8.0_282-b08
Test Results	https://ci-hbase.apache.org/job/HBase-Connectors-PreCommit/job/PR-133/1/testReport/
Max. process+thread count	43 (vs. ulimit of 12500)
modules	C: kafka/hbase-kafka-proxy U: kafka/hbase-kafka-proxy
Console output	https://ci-hbase.apache.org/job/HBase-Connectors-PreCommit/job/PR-133/1/console
versions	git=2.20.1
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[NihalJain]

Gentle ping @Apache9

[stoty]

+1 LGTM

[ndimiduk]

This is a fragile approach. This repository needs sureing up if it is going to intentionally support multiple minor versions of HBase. At the minimum, we need an HBase versions check in our pre-commit and nightlies, similar to what HBase has for Hadoop.

[stoty]

Could be modelled on the HBase profiles setup in Phoenix.

[Matulis]

@NihalJain could you possibly share the dependency versions you were using to build with HBase 2.6.0? I'm actually trying to get the package to build for another HBase version, but was running into problems with the Hadoop version, 3.2.4 worked (when using HBase 2.5.10), but nothing above that, just wondering what combination ended up working for HBase 2.6.0?

[NihalJain]

I have built with hadoop 3.3.6 apart from 3.2.4. Which are the build versions you are using? What is the issue? Mind sharing may be I can take a look

[Matulis]

@NihalJain thanks for responding so quickly! Here is the branch I am testing: https://github.com/Matulis/hbase-connectors/tree/WIP_Hadoop_3_4_0

I'm trying to pin the versions to match AWS EMR 7.5.0.

It looks like the HBase cluster fails to start (I see the same with 3.3.6), not sure if it's possibly related to building on ARM based Mac (I'll try to get someone on an Intel Mac to try the build):

[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 35.342 s <<< FAILURE! - in org.apache.hadoop.hbase.spark.TestJavaHBaseContextForLargeRows
[ERROR] org.apache.hadoop.hbase.spark.TestJavaHBaseContextForLargeRows  Time elapsed: 35.342 s  <<< ERROR!
java.io.IOException: Shutting down
	at org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:257)
	at org.apache.hadoop.hbase.MiniHBaseCluster.<init>(MiniHBaseCluster.java:110)
	at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:1185)
	at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:1152)
	at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:1116)
	at org.apache.hadoop.hbase.spark.TestJavaHBaseContext.init(TestJavaHBaseContext.java:107)
	at org.apache.hadoop.hbase.spark.TestJavaHBaseContextForLargeRows.setUpBeforeClass(TestJavaHBaseContextForLargeRows.java:51)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
	at org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
	at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.RuntimeException: Master not active after 30000ms
	at org.apache.hadoop.hbase.util.JVMClusterUtil.waitForEvent(JVMClusterUtil.java:221)
	at org.apache.hadoop.hbase.util.JVMClusterUtil.startup(JVMClusterUtil.java:177)
	at org.apache.hadoop.hbase.LocalHBaseCluster.startup(LocalHBaseCluster.java:407)
	at org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:250)
	... 20 more

[NihalJain]

@NihalJain thanks for responding so quickly! Here is the branch I am testing: https://github.com/Matulis/hbase-connectors/tree/WIP_Hadoop_3_4_0

I'm trying to pin the versions to match AWS EMR 7.5.0.

It looks like the HBase cluster fails to start (I see the same with 3.3.6), not sure if it's possibly related to building on ARM based Mac (I'll try to get someone on an Intel Mac to try the build):

[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 35.342 s <<< FAILURE! - in org.apache.hadoop.hbase.spark.TestJavaHBaseContextForLargeRows
[ERROR] org.apache.hadoop.hbase.spark.TestJavaHBaseContextForLargeRows  Time elapsed: 35.342 s  <<< ERROR!
java.io.IOException: Shutting down
	at org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:257)
	at org.apache.hadoop.hbase.MiniHBaseCluster.<init>(MiniHBaseCluster.java:110)
	at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:1185)
	at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:1152)
	at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:1116)
	at org.apache.hadoop.hbase.spark.TestJavaHBaseContext.init(TestJavaHBaseContext.java:107)
	at org.apache.hadoop.hbase.spark.TestJavaHBaseContextForLargeRows.setUpBeforeClass(TestJavaHBaseContextForLargeRows.java:51)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
	at org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
	at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.RuntimeException: Master not active after 30000ms
	at org.apache.hadoop.hbase.util.JVMClusterUtil.waitForEvent(JVMClusterUtil.java:221)
	at org.apache.hadoop.hbase.util.JVMClusterUtil.startup(JVMClusterUtil.java:177)
	at org.apache.hadoop.hbase.LocalHBaseCluster.startup(LocalHBaseCluster.java:407)
	at org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:250)
	... 20 more

Thanks for the response. I think the issue is mockito. You have 2 options:

• Try to fork PR HBASE-28382 Support building hbase-connectors with JDK17 #132 and apply your changes on top this. This PR is ready only; I could not get it merged as during review some additional work around builds was requested and just forgot about it. But those are not really imp if you have your own build process.
• Or just try cherry-pick changes for mockito from above PR.

Let me know if this helps.

[Matulis]

@NihalJain so I pulled your PR (with no changes it builds), when I try to bump the Hadoop version to 3.3.6 I get the same issue. Do you remember which permutation of dependencies work on 3.3.6 for you? I'm currently using JDK8, but will try with 17 to see if anything changes. Thanks!