[SPARK-14807] Create a compatibility module

[yhuai]

What changes were proposed in this pull request?

This PR creates a compatibility module in sql (called hive-1-x-compatibility), which will host HiveContext in Spark 2.0 (moving HiveContext to here will be done separately). This module is not included in assembly because only users who still want to access HiveContext need it.

How was this patch tested?

I manually tested sbt/sbt -Phive package and mvn -Phive package -DskipTests.

[rxin]

add an extra line

[rxin]

I'd name this something like spark-1.x-compatibility (is dot allowed in the name?)

[yhuai]

Looks like we should move sketch and compatibility out from this list once we release 2.0, right?

[JoshRosen]

Yes. The same goes for sketch and mllibLocal. Can you file a blocker JIRA for 2.0.1 / 2.1 to make sure that we audit this post-release?

[rxin]

also this should probably go into sql?

[SparkQA]

Test build #56572 has finished for PR 12580 at commit 30e541d.

• This patch fails Scala style tests.
• This patch merges cleanly.
• This patch adds no public classes.

[JoshRosen]

We might want to pre-emptively add hive to this list.

[srowen]

Broadly, why is this a new and generic module? it seems quite Hive specific. I get that other fairly different compatibility shims could go here later, but those would properly be different modules right?

[yhuai]

@srowen I am moving this into sql.

[SparkQA]

Test build #56578 has started for PR 12580 at commit 7777ffb.

[yhuai]

@JoshRosen https://issues.apache.org/jira/browse/SPARK-14818 is the jira for updating mima exclusions.

[yhuai]

@srowen @rxin @JoshRosen How's this version?

[SparkQA]

Test build #56580 has finished for PR 12580 at commit 92331f3.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[holdenk]

Supressing warnings for sun.misc.unsafe sounds good, but do we need it in the hive compatibility shims project (this does suppress other warnings that we might want to see if we aren't planning on using sun.misc.unsafe here)?

[SparkQA]

Test build #56604 has finished for PR 12580 at commit 3514b3c.

• This patch fails executing the dev/run-tests script.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #56606 has started for PR 12580 at commit 166b34a.

[SparkQA]

Test build #56608 has finished for PR 12580 at commit 8d54be3.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[JoshRosen]

LGTM. We can always change the name later.