Skip to content

Prioritize high confidence stats during broadcast joins#23016

Merged
abhinavmuk04 merged 1 commit intoprestodb:masterfrom
abhinavmuk04:milestone2p1
Jul 1, 2024
Merged

Prioritize high confidence stats during broadcast joins#23016
abhinavmuk04 merged 1 commit intoprestodb:masterfrom
abhinavmuk04:milestone2p1

Conversation

@abhinavmuk04
Copy link
Contributor

@abhinavmuk04 abhinavmuk04 commented Jun 14, 2024

Description

Prioritize high confidence stats during broadcast joins if enabled

Motivation and Context

When there are two PlanNodes in which they are both small enough for broadcast join we will prioritize the side which has higher confidence stats. If they both have high confidence stats then we keep the original behavior. The user has the ability to turn this on and off.

Impact

This change will create a feature which the user can utilize to improve optimization and help improve the execution time of broadcast join queries

Test Plan

Implemented various tests in both DetermineJoinDistributionType and ReorderJoinsType, which will check if, with the session property enable, nodes with the higher confidence stats will be broadcasted

Contributor checklist

  • Please make sure your submission complies with our development, formatting, commit message, and attribution guidelines.
  • PR description addresses the issue accurately and concisely. If the change is non-trivial, a GitHub Issue is referenced.
  • Documented new properties (with its default value), SQL syntax, functions, or other functionality.
  • If release notes are required, they follow the release notes guidelines.
  • Adequate tests were added if applicable.
  • CI passed.

Release Notes

General Changes
* Add confidence based broadcasting, side of join with highest confidence will be on build side. 
  This can be enabled with the ``confidence_based_broadcast`` session property :pr:`23016`

@abhinavmuk04 abhinavmuk04 changed the title Prioritize HBO stats during broadcast joins Prioritize high confidence stats during broadcast joins Jun 27, 2024
@abhinavmuk04 abhinavmuk04 marked this pull request as ready for review June 27, 2024 19:55
@abhinavmuk04 abhinavmuk04 requested review from a team, feilong-liu and jaystarshot as code owners June 27, 2024 19:55
@abhinavmuk04 abhinavmuk04 requested a review from presto-oss June 27, 2024 19:55
@abhinavmuk04 abhinavmuk04 requested a review from feilong-liu June 30, 2024 04:30
@abhinavmuk04 abhinavmuk04 requested a review from feilong-liu July 1, 2024 21:15
@feilong-liu
Copy link
Contributor

Code lgtm. However, as a code owner, I do not have ownership for the SystemSessionProperty file, will need an committer approval for help to merge the change here.

@abhinavmuk04 abhinavmuk04 merged commit 461a31a into prestodb:master Jul 1, 2024
@abhinavmuk04 abhinavmuk04 deleted the milestone2p1 branch July 1, 2024 22:38
@tdcmeehan tdcmeehan mentioned this pull request Aug 23, 2024
34 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants