Skip to content

[SPARK-4925][SQL] Publish Spark SQL hive-thriftserver maven artifact #3766

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
alexliu68 wants to merge 2 commits into from
Closed

[SPARK-4925][SQL] Publish Spark SQL hive-thriftserver maven artifact #3766

alexliu68 wants to merge 2 commits into from

Conversation

@alexliu68
Copy link

@alexliu68 alexliu68 commented Dec 23, 2014

No description provided.

@AmplabJenkins
Copy link

AmplabJenkins commented Dec 23, 2014

Can one of the admins verify this patch?

@vanzin
Copy link
Contributor

vanzin commented Dec 23, 2014

This doesn't really seem related to SPARK-3816...

@pwendell
Copy link
Contributor

pwendell commented Dec 23, 2014

I commented on the JIRA - but we removed this intentionally. It would be helpful to understand why you are linking against this in Maven. It's typically used when building Spark distributions.

@pwendell
Copy link
Contributor

pwendell commented Dec 23, 2014

The correct JIRA is https://issues.apache.org/jira/browse/SPARK-4925

@alexliu68 alexliu68 changed the title [SPARK-3816][SQL] Publish Spark SQL hive-thriftserver maven artifact [SPARK-4925[SQL] Publish Spark SQL hive-thriftserver maven artifact Dec 23, 2014
@alexliu68 alexliu68 changed the title [SPARK-4925[SQL] Publish Spark SQL hive-thriftserver maven artifact [SPARK-4925][SQL] Publish Spark SQL hive-thriftserver maven artifact Dec 23, 2014
@alexliu68
Copy link
Author

alexliu68 commented Dec 23, 2014

We integrate Spark hive-thriftserver with Cassandra by downloading the hive-thriftserver by build.xml and copying it to class path. Currently/temporarily we publish it to our private repository, but we hope we don't need maintain the artifact at our private repo and have to build it each time for a new release. Publish it to public maven repository will help a lot for us.

@liancheng
Copy link
Contributor

liancheng commented Dec 23, 2014

@pwendell Would like to add another (not so sound) reason for a user to link against the Thrift server, which is the HiveThriftServer2.startWithContext method. Users can use this to serve tables created from plain RDDs via JDBC. However this feature is quite experimental and doesn't play well with other parts, for example, SHOW TABLE doesn't show those RDD tables.

pwendell
pwendell reviewed Dec 27, 2014
View reviewed changes
pom.xml Outdated
Copy link
Contributor

@pwendell pwendell Dec 27, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be removed - we only want this enabled with the -Phive-thriftserver profile. We always enable that profile when publishing artifacts.

@pwendell
Copy link
Contributor

pwendell commented Dec 27, 2014

Okay - makes sense. There is one incorrect change in here, but once that's removed we can merge this.

@alexliu68
Copy link
Author

alexliu68 commented Dec 30, 2014

I removed it.

@marmbrus
Copy link
Contributor

marmbrus commented Jan 7, 2015

ok to test

@SparkQA
Copy link

SparkQA commented Jan 7, 2015

Test build #25183 has started for PR 3766 at commit 3137b51.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Jan 8, 2015

Test build #25183 has finished for PR 3766 at commit 3137b51.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

AmplabJenkins commented Jan 8, 2015

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25183/
Test PASSed.

asfgit pushed a commit that referenced this pull request Jan 10, 2015
@alexoss68 @marmbrus
[SPARK-4925][SQL] Publish Spark SQL hive-thriftserver maven artifact
09eef3b 
Author: Alex Liu <[email protected]>

Closes #3766 from alexliu68/SPARK-SQL-4925 and squashes the following commits:

3137b51 [Alex Liu] [SPARK-4925][SQL] Remove sql/hive-thriftserver module from pom.xml
15f2e38 [Alex Liu] [SPARK-4925][SQL] Publish Spark SQL hive-thriftserver maven artifact

(cherry picked from commit 1e56eba)
Signed-off-by: Michael Armbrust <[email protected]>
@marmbrus
Copy link
Contributor

marmbrus commented Jan 10, 2015

Thanks! Merged to master and branch-1.2. @pwendell is going to publish sometime later this weekend.

@asfgit asfgit closed this in 1e56eba Jan 10, 2015
@nettok
Copy link

nettok commented Mar 4, 2015

I can't find the artifact in http://mvnrepository.com/artifact/org.apache.spark

Any reason why it hasn't been published yet?

@stau
Copy link

stau commented Mar 17, 2015

It probably got overlooked. The module needs to be published manually since it's not a default module in the root pom.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

10 participants

@alexliu68 @AmplabJenkins @vanzin @pwendell @liancheng @marmbrus @SparkQA @nettok @stau @alexoss68