Skip to content

Conversation

@LucaCanali
Copy link
Contributor

Queries with filters may error out with ClassNotFoundException as detailed in HBASE-22769.
The error can be worked around with .option("hbase.spark.pushdown.columnfilter", false) but this bypasses SQL pushdown, which is a useful feature.
The error appears to come from reflection. This PR aims at making reflection explicit for org.apache.hadoop.hbase.spark.datasources.JavaBytesEncoder and org.apache.hadoop.hbase.spark.DynamicLogicExpressionBuilder.
The patch has been manually tested.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 9s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 spotbugs 0m 0s spotbugs executables are not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-0 ⚠️ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ master Compile Tests _
+1 💚 mvninstall 1m 38s master passed
+1 💚 compile 0m 36s master passed
+1 💚 javadoc 0m 9s master passed
_ Patch Compile Tests _
+1 💚 mvninstall 0m 47s the patch passed
+1 💚 compile 0m 37s the patch passed
+1 💚 javac 0m 37s the patch passed
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 javadoc 0m 7s the patch passed
_ Other Tests _
+1 💚 unit 3m 56s hbase-spark in the patch passed.
9m 7s
Subsystem Report/Notes
Docker ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-Connectors-PreCommit/job/PR-74/1/artifact/yetus-precommit-check/output/Dockerfile
GITHUB PR #74
Optional Tests dupname javac javadoc unit spotbugs findbugs compile
uname Linux 178e1770a153 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 GNU/Linux
Build tool hb_maven
Personality dev-support/jenkins/hbase-personality.sh
git revision master / b9706c8
Default Java Oracle Corporation-1.8.0_275-b01
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-Connectors-PreCommit/job/PR-74/1/testReport/
Max. process+thread count 890 (vs. ulimit of 12500)
modules C: spark/hbase-spark U: spark/hbase-spark
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-Connectors-PreCommit/job/PR-74/1/console
versions git=2.20.1
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@LucaCanali LucaCanali closed this Nov 26, 2020
@LucaCanali LucaCanali deleted the sqlpushdownWithFilter branch November 26, 2020 10:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants