Skip to content

Conversation

@lichenglin
Copy link
Contributor

What is this PR for?

remove org.apache.spark.sql.hive.HiveSharedState class check
because it has been removed since spark 2.1.x

What type of PR is it?

[Bug Fix ]

What is the Jira issue?

ZEPPELIN-1909

remove org.apache.spark.sql.hive.HiveSharedState class check .

Because this class has been removed from spark since 2.1.0
https://issues.apache.org/jira/browse/ZEPPELIN-1909
make hive enable under spark 2.1.0
@felixcheung
Copy link
Member

looks good, why doesn't CI tests for Spark 2.1 fails for this?

@felixcheung
Copy link
Member

would be great to understand why it worked before, but might merge this fix first.
merging if no more comment

@lichenglin
Copy link
Contributor Author

I don't know either,It fails with spark 1.6.1.
May be the class check is still needed by 1.6,1?I'm not sure.
How can I trigger the build once more on travis-cli?

@felixcheung
Copy link
Member

I don't think we need to check for all 3 classes, so I believe this fix is good.
what I don't understand is why it is failing for you but running just fine in travis CI where we have tests for Spark 2.1 - which is why I'm asking.

to trigger test, close this PR and reopen it.

@lichenglin
Copy link
Contributor Author

e...
It works on my local mac with spark 2.1.0.
And pass the build with 2.1.0.
I seems that the build fails with
{code}
"env": "PYTHON="2" SCALA_VER="2.10" SPARK_VER="1.6.1" HADOOP_VER="2.6" PROFILE="-Pspark-1.6 -Phadoop-2.6 -Ppyspark" BUILD_FLAG="package -pl spark,python -am -DskipTests -DskipRat" TEST_FLAG="verify -DskipRat" TEST_PROJECTS="-pl zeppelin-interpreter,zeppelin-display,spark-dependencies,spark,python -Dtest=org.apache.zeppelin.spark.PySparkTest,org.apache.zeppelin.python. -Dpyspark.test.exclude='' -DfailIfNoTests=false"",
{code}

I think we'd better reopen the PR and build once more.
Maybe the class check is still necessary for spark 1.6.1

@felixcheung
Copy link
Member

it's only the python2 + spark 1.6 profile. It ran fine with the other spark 1.6 profile so I think we are good.

@asfgit asfgit closed this in 0f65714 Jan 7, 2017
beriaanirudh pushed a commit to beriaanirudh/zeppelin that referenced this pull request Mar 8, 2017
### What is this PR for?
remove org.apache.spark.sql.hive.HiveSharedState class check
because it has been removed since spark 2.1.x

### What type of PR is it?
[Bug Fix ]

### What is the Jira issue?
[ZEPPELIN-1909](https://issues.apache.org/jira/browse/ZEPPELIN-1909)

Author: lichenglin <[email protected]>

Closes apache#1856 from lichenglin/master and squashes the following commits:

60c3c04 [lichenglin] Merge pull request apache#1 from lichenglin/lichenglin-patch-1
3f90485 [lichenglin] make hive enable under spark 2.1.0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants