Skip to content

Conversation

@cloud-fan
Copy link
Contributor

It's a follow up of #6173, for expressions like Coalesce that have a Seq[Expression], when we do semantic equal check for it, we need to do semantic equal check for all of its children.
Also we can just use Seq[(Expression, NamedExpression)] instead of Map[Expression, NamedExpression] as we only search it with find.

@chenghao-intel, I agree that we probably never knows semanticEquals in a general way, but I think we have done that in TreeNode, so we can use similar logic. Then we can handle something like Coalesce(children: Seq[Expression]) correctly.

@AmplabJenkins
Copy link

Merged build triggered.

@AmplabJenkins
Copy link

Merged build started.

@SparkQA
Copy link

SparkQA commented May 19, 2015

Test build #33077 has started for PR 6261 at commit 23491da.

@SparkQA
Copy link

SparkQA commented May 19, 2015

Test build #33077 has finished for PR 6261 at commit 23491da.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

Merged build finished. Test PASSed.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33077/
Test PASSed.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this for expressions like Coalesce whose children order is sensitive?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not about the order, it's just we don't need a map here as we search it with find, not get.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I see. Makes sense.

@liancheng
Copy link
Contributor

@cloud-fan Would you mind to update the PR description so that it briefly summaries changes made in this change, and add the current PR description as a comment?

@AmplabJenkins
Copy link

Merged build triggered.

@AmplabJenkins
Copy link

Merged build started.

@SparkQA
Copy link

SparkQA commented Jun 10, 2015

Test build #34604 has started for PR 6261 at commit 7f794cd.

@SparkQA
Copy link

SparkQA commented Jun 10, 2015

Test build #34604 has finished for PR 6261 at commit 7f794cd.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

Merged build finished. Test FAILed.

@AmplabJenkins
Copy link

Merged build triggered.

@AmplabJenkins
Copy link

Merged build started.

@SparkQA
Copy link

SparkQA commented Jun 12, 2015

Test build #34747 has started for PR 6261 at commit 4daef88.

@SparkQA
Copy link

SparkQA commented Jun 12, 2015

Test build #34747 has finished for PR 6261 at commit 4daef88.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

Merged build finished. Test PASSed.

@cloud-fan
Copy link
Contributor Author

cc @marmbrus @liancheng

@liancheng
Copy link
Contributor

Thanks, merging to master.

@asfgit asfgit closed this in c19c785 Jun 12, 2015
@liancheng
Copy link
Contributor

Forgot to mention before merging this... But please always file a JIRA ticket to track PRs, unless the PR is really minor updates (like documentation/comment typos).

@cloud-fan
Copy link
Contributor Author

sorry... will keep it in mind :)

@cloud-fan cloud-fan deleted the tmp branch June 12, 2015 09:06
nemccarthy pushed a commit to nemccarthy/spark that referenced this pull request Jun 19, 2015
It's a follow up of apache#6173, for expressions like `Coalesce` that have a `Seq[Expression]`, when we do semantic equal check for it, we need to do semantic equal check for all of its children.
Also we can just use `Seq[(Expression, NamedExpression)]` instead of `Map[Expression, NamedExpression]` as we only search it with `find`.

chenghao-intel, I agree that we probably never knows `semanticEquals` in a general way, but I think we have done that in `TreeNode`, so we can use similar logic. Then we can handle something like `Coalesce(children: Seq[Expression])` correctly.

Author: Wenchen Fan <[email protected]>

Closes apache#6261 from cloud-fan/tmp and squashes the following commits:

4daef88 [Wenchen Fan] address comments
dd8fbd9 [Wenchen Fan] correct semanticEquals
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants