Enhance join reordering to work with non-simple equi-join clauses

[aaneja]

Description

Enhance join-reordering to work with non-simple equi-join predicates

Motivation and Context

Reapplies #20413 with fixes to bugs identified -

1. The output variables of JoinNode could be referring to a variable generated from an intermediate Project. The output variables of the CanonicalJoinNode were not being updated to reflect this correctly
2. Intermediate assignments tracked while flattening the ProjectNodes should be recursively falttened so that the CanonicalJoinNode only refers to input/output variables from the actual join sources

Additionally, the feature config for optimizer.handle-complex-equi-joins was changed to a default false to allow testing this in isolation/rollout. We will set this to default true in the future, once we test with more workloads

Test Plan

New tests added to address the bug identified -

1. testProjectNodesBetweenJoinNodesAreFlattenedForComplexEquiJoins was rewritten to include multiple ProjectNode's (including nested Projects and a simple rename projection). Additionally, further assertions were added on the overall assignments built and the output variables that were rewritten
2. testComplexEquiJoinCriteriaForJoinsWithUSINGClause was added which runs a SELECT * query with a USING clause

Contributor checklist

• Please make sure your submission complies with our development, formatting, commit message, and attribution guidelines.
• PR description addresses the issue accurately and concisely. If the change is non-trivial, a GitHub Issue is referenced.
• Documented new properties (with its default value), SQL syntax, functions, or other functionality.
• If release notes are required, they follow the release notes guidelines.
• Adequate tests were added if applicable.
• CI passed.

Release Notes

Please follow release notes guidelines and fill in the release notes below.

== RELEASE NOTES ==

General Changes
* Enhance join-reordering to work with non-simple equi-join predicates

[vivek-bharathan]

is this unrelated?

[aaneja]

Yes. I noticed that the cherry-picked code had this added to ensure that we always run this test with no join ordering. This was missed when we pulled it into Presto.

[feilong-liu]

Can you highlight the change you added compared with previous PR (maybe by adding PR comments for the code) to help review?

[aaneja]

@feilong-liu Here are the delta of the changes from the previous PR

[aaneja]

The core of the changes from the previous PR are to this class

[aaneja]

We recursively resolve all intermediate assignments to only use the input variables of the join sources, e.g:

inter_3 = inter_2
inter_2 = inter_1 + join_op_1
inter_1 = join_op_2 + join_op_0

is rewritten to :

inter_3 = join_op_2 + join_op_0 + join_op_1
inter_2 = join_op_2 + join_op_0 + join_op_1
inter_1 = join_op_2 + join_op_0

[aaneja]

While building the MultiJoinNode, we take care to set the output variables of the MultiJoinNode to only refer to the input variables.
Any output variables that were using variables from the intermediate project nodes are tracked in overallAssigments
More details are in code comments

[aaneja]

I modified the test to demonstrate how multiple nested Project's are flattened

[aaneja]

renamePlusSum is a variable from an intermediate Project - this gets set on the overall assignment of the MultiJoinNode with the resolved variables of the input sources