-
Notifications
You must be signed in to change notification settings - Fork 5.5k
Add an optimizer to add distinct below build side of semi join #25238
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add an optimizer to add distinct below build side of semi join #25238
Conversation
6d6ebc0 to
9ce3afa
Compare
9ce3afa to
a6095f6
Compare
...rc/main/java/com/facebook/presto/sql/planner/iterative/rule/AddDistinctForSemiJoinBuild.java
Show resolved
Hide resolved
a6095f6 to
9f67498
Compare
9f67498 to
52819a6
Compare
| private boolean nativeExecutionTypeRewriteEnabled; | ||
| private String expressionOptimizerName = DEFAULT_EXPRESSION_OPTIMIZER_NAME; | ||
| private boolean addExchangeBelowPartialAggregationOverGroupId; | ||
| private boolean addDistinctBelowSemiJoinBuild; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This can be on by default?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd like to run it for some time before turning it on as default.
| import static com.facebook.presto.spi.plan.AggregationNode.isDistinct; | ||
| import static com.facebook.presto.spi.plan.AggregationNode.singleGroupingSet; | ||
| import static com.facebook.presto.sql.planner.plan.Patterns.semiJoin; | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please add small comment explaining the rule
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Description
We found that adding a distinct aggregation before a semi join on the build side can improve latency. This is probably due to the restriction that the build side of semi join is executed with single thread
presto/presto-main-base/src/main/java/com/facebook/presto/sql/planner/LocalExecutionPlanner.java
Line 2623 in a12b523
This PR works by adding a distinct aggregation below the semi join on the build side.
Motivation and Context
To improve latency of semi join
Impact
To improve latency of semi join
Test Plan
Unit tests
Contributor checklist
Release Notes
Please follow release notes guidelines and fill in the release notes below.