Skip to content

[SPARK-33473][SQL] Extend interpreted subexpression elimination to other interpreted projections #30406

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
viirya wants to merge 3 commits into from
Closed

[SPARK-33473][SQL] Extend interpreted subexpression elimination to other interpreted projections #30406

viirya wants to merge 3 commits into from

Conversation

@viirya
Copy link
Member

@viirya viirya commented Nov 18, 2020

What changes were proposed in this pull request?

Similar to InterpretedUnsafeProjection, this patch proposes to extend interpreted subexpression elimination to InterpretedMutableProjection and InterpretedSafeProjection.

Why are the changes needed?

Enabling subexpression elimination can improve the performance of interpreted projections, as shown in InterpretedUnsafeProjection.

Does this PR introduce any user-facing change?

No

How was this patch tested?

Unit test.

@maropu
Copy link
Member

maropu commented Nov 18, 2020

Looks fine if the tests pass.

@github-actions github-actions bot added the SQL label Nov 18, 2020
@SparkQA
Copy link

SparkQA commented Nov 18, 2020

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35860/

@SparkQA
Copy link

SparkQA commented Nov 18, 2020

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35861/

@SparkQA

This comment has been minimized.

Sign in to view
@SparkQA

This comment has been minimized.

Sign in to view
@viirya
Copy link
Member Author

viirya commented Nov 18, 2020

retest this please

@SparkQA
Copy link

SparkQA commented Nov 18, 2020

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35860/

@SparkQA
Copy link

SparkQA commented Nov 18, 2020

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35861/

@SparkQA

This comment has been minimized.

Sign in to view
@SparkQA
Copy link

SparkQA commented Nov 18, 2020

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35875/

@SparkQA
Copy link

SparkQA commented Nov 18, 2020

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35875/

@viirya
Copy link
Member Author

viirya commented Nov 18, 2020

retest this please

@SparkQA
Copy link

SparkQA commented Nov 18, 2020

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35898/

@SparkQA
Copy link

SparkQA commented Nov 18, 2020

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35898/

@SparkQA
Copy link

SparkQA commented Nov 18, 2020

Test build #131293 has finished for PR 30406 at commit ee4b070.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

dongjoon-hyun
dongjoon-hyun approved these changes Nov 19, 2020
View reviewed changes
Copy link
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, LGTM. Thank you, @viirya and @maropu .
Merged to master for Apache Spark 3.1.

@viirya
Copy link
Member Author

viirya commented Nov 19, 2020

Thanks @maropu @dongjoon-hyun

@viirya viirya deleted the SPARK-33473 branch December 27, 2023 18:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@maropu maropu maropu approved these changes

@dongjoon-hyun dongjoon-hyun dongjoon-hyun approved these changes

Assignees

No one assigned

Labels

SQL

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@viirya @maropu @SparkQA @dongjoon-hyun