Skip to content

Commit c07838b

Browse files
dennishuomarmbrus
authored andcommitted
[SPARK-9206] [SQL] Fix HiveContext classloading for GCS connector.
IsolatedClientLoader.isSharedClass includes all of com.google.\*, presumably for Guava, protobuf, and/or other shared Google libraries, but needs to count com.google.cloud.\* as "hive classes" when determining which ClassLoader to use. Otherwise, things like HiveContext.parquetFile will throw a ClassCastException when fs.defaultFS is set to a Google Cloud Storage (gs://) path. On StackOverflow: http://stackoverflow.com/questions/31478955 EDIT: Adding yhuai who worked on the relevant classloading isolation pieces. Author: Dennis Huo <[email protected]> Closes #7549 from dennishuo/dhuo-fix-hivecontext-gcs and squashes the following commits: 1f8db07 [Dennis Huo] Fix HiveContext classloading for GCS connector.
1 parent 60c0ce1 commit c07838b

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

sql/hive/src/main/scala/org/apache/spark/sql/hive/client/IsolatedClientLoader.scala

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -125,7 +125,7 @@ private[hive] class IsolatedClientLoader(
125125
name.contains("log4j") ||
126126
name.startsWith("org.apache.spark.") ||
127127
name.startsWith("scala.") ||
128-
name.startsWith("com.google") ||
128+
(name.startsWith("com.google") && !name.startsWith("com.google.cloud")) ||
129129
name.startsWith("java.lang.") ||
130130
name.startsWith("java.net") ||
131131
sharedPrefixes.exists(name.startsWith)

0 commit comments

Comments
 (0)