Add null stats for join probe

[feilong-liu]

Description

Part of #20355

Track number of probe keys and number of null probe keys in HBO, and use it to enable outer join null salt.

Motivation and Context

Currently HBO only records the null key stats for join build side, and use it to enable null salt in HBO. This PR also add probe side information and use it to enable null salt in HBO too.

Impact

Resolve the probe side skew in null join.

Test Plan

Add unit test
And end to end test on tracking

Contributor checklist

• Please make sure your submission complies with our development, formatting, commit message, and attribution guidelines.
• PR description addresses the issue accurately and concisely. If the change is non-trivial, a GitHub Issue is referenced.
• Documented new properties (with its default value), SQL syntax, functions, or other functionality.
• If release notes are required, they follow the release notes guidelines.
• Adequate tests were added if applicable.
• CI passed.

Release Notes

Please follow release notes guidelines and fill in the release notes below.

== RELEASE NOTES ==

General Changes
* Track null probe keys in join and use it to enable outer join null salt in history based optimization.

[feilong-liu]

Piggyback the count here, without adding overhead in counting.

[kaikalur]

Looks good.

High level observation - salting on the inner side (right side of left and left side of right) is unnecessary. We should instead be filtering them out.

[feilong-liu]

Looks good.

High level observation - salting on the inner side (right side of left and left side of right) is unnecessary. We should instead be filtering them out.

Sure, I think this should be in a separate rule, and with this null filter, we will not hit the null skew in the inner side. Looks like that we already have one such rule AddNotNullFiltersToJoinNode which is default to false and we need to enable it.

[kaikalur]

[like] Sreeni Viswanadha reacted to your message:

…

________________________________ From: feilong-liu ***@***.***> Sent: Tuesday, October 24, 2023 10:08:09 PM To: prestodb/presto ***@***.***> Cc: Sreeni Viswanadha ***@***.***>; Review requested ***@***.***> Subject: Re: [prestodb/presto] Add null stats for join probe (PR #21203) Looks good. High level observation - salting on the inner side (right side of left and left side of right) is unnecessary. We should instead be filtering them out. Sure, I think this should be in a separate rule, and with this null filter, ZjQcmQRYFpfptBannerStart This Message Is From an External Sender ZjQcmQRYFpfptBannerEnd Looks good. High level observation - salting on the inner side (right side of left and left side of right) is unnecessary. We should instead be filtering them out. Sure, I think this should be in a separate rule, and with this null filter, we will not hit the null skew in the inner side. Looks like that we already have one such rule AddNotNullFiltersToJoinNode which is default to false and we need to enable it. — Reply to this email directly, view it on GitHub<#21203 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAPNYANMLVON7NRSPQSLDOTYBA34TAVCNFSM6AAAAAA6JQ6K7KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONZYGEZDCMBRGU>. You are receiving this because your review was requested.Message ID: ***@***.***>

[pranjalssh]

Should this be based on ratio as compared to counts?

[kaikalur]

Yeah but counts are better because irrespective of how big the table is you have to shuffle these anyway so in some sense it's not related to ratio per se