Skip to content

[SPARK-6555] [SQL] Overrides equals() and hashCode() for MetastoreRelation #5289

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
liancheng wants to merge 3 commits into from
Closed

[SPARK-6555] [SQL] Overrides equals() and hashCode() for MetastoreRelation #5289

liancheng wants to merge 3 commits into from

Conversation

@liancheng
Copy link
Contributor

@liancheng liancheng commented Mar 31, 2015

Also removes temporary workarounds made in #5183 and #5251.

Review on Reviewable

@SparkQA
Copy link

SparkQA commented Mar 31, 2015

Test build #29472 has started for PR 5289 at commit cfafeeb.

liancheng
liancheng reviewed Mar 31, 2015
View reviewed changes
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HivePlanTest.scala
Copy link
Contributor Author

@liancheng liancheng Mar 31, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually the trouble maker is PlanTest.comparePlans, because now MetastoreRelation also compares output attributes. But right now I don't have a clean solution for this. Thus, I replace the original metastore relation with a temporary relation to pass the test here.

Copy link
Contributor

@marmbrus marmbrus Mar 31, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think thats actually reasonable. Compare plans is mostly about checking plan structure.

Copy link
Contributor

@marmbrus marmbrus Mar 31, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd even be okay adding a special rule to compare plans that always replaces MetastoreRelations with LocalRelation

@SparkQA
Copy link

SparkQA commented Mar 31, 2015

Test build #29474 has started for PR 5289 at commit d0095ac.

@SparkQA
Copy link

SparkQA commented Mar 31, 2015

Test build #29472 has finished for PR 5289 at commit cfafeeb.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • case class AtLeastNNonNulls(n: Int, children: Seq[Expression]) extends Predicate
  • This patch does not change any dependencies.

@AmplabJenkins
Copy link

AmplabJenkins commented Mar 31, 2015

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29472/
Test PASSed.

@SparkQA
Copy link

SparkQA commented Mar 31, 2015

Test build #29474 has finished for PR 5289 at commit d0095ac.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.
  • This patch does not change any dependencies.

@AmplabJenkins
Copy link

AmplabJenkins commented Mar 31, 2015

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29474/
Test PASSed.

liancheng
liancheng reviewed Mar 31, 2015
View reviewed changes
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala
Copy link
Contributor Author

@liancheng liancheng Mar 31, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All changes above this line are about reverting workaround introduced in #5183.

asfgit pushed a commit that referenced this pull request Mar 31, 2015
@liancheng @marmbrus
[SPARK-6555] [SQL] Overrides equals() and hashCode() for MetastoreRel…
9ebefb1 
…ation

Also removes temporary workarounds made in #5183 and #5251.

<!-- Reviewable:start -->
[<img src="https://reviewable.io/review_button.png" height=40 alt="Review on Reviewable"/>](https://reviewable.io/reviews/apache/spark/5289)
<!-- Reviewable:end -->

Author: Cheng Lian <[email protected]>

Closes #5289 from liancheng/spark-6555 and squashes the following commits:

d0095ac [Cheng Lian] Removes unused imports
cfafeeb [Cheng Lian] Removes outdated comment
75a2746 [Cheng Lian] Overrides equals() and hashCode() for MetastoreRelation

(cherry picked from commit a7992ff)
Signed-off-by: Michael Armbrust <[email protected]>
@asfgit asfgit closed this in a7992ff Mar 31, 2015
@marmbrus
Copy link
Contributor

marmbrus commented Mar 31, 2015

Thanks! Merged to master and 1.3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@liancheng @SparkQA @AmplabJenkins @marmbrus