A new rule engine for Optimizer

[elsa0520]

Feature Request

Is your feature request related to a problem? Please describe:
Current optimizer rewrite rules are executed sequentially once and some specific rules (column prunning) are done multiple times hard coded.

• Applying rules multiple times is not possible if a rule triggers another rule (for example: eliminating projections triggers join elimination). Missing such cases are lost optimization opportunities.

Describe the feature you'd like:

The New rule engine for pre-processing phase changes logical plan to another logical plan.
It also should satisfy following requirements:

• Applying rules multiple times by rule engine instead of hard code manually.
• Clean fast path for fast queries
• Easier testing for enabling/disabling rules
• Adding new rules is easier with this new architecture

Design

Design

• Code factoring:
  • Functional components (FD, derived statistics and hint) out of logical plan building. The Rule Engine: refactor of Functional components #43361
  • Make sure each optimization rewrite has no side effects. If any, remove the side and apply it as a functional rewrite.
• Rule engine design is based on the following abstraction
  • List of API's for rule1, rule2, ... RuleN. (ordered rule set as set of interfaces)
  • Order of rules "RuleOrder" = {RuleX, RuleY, ....}
  • 2 dimensional array of rule interactions where M(i,j) = true means that If Rulei is applied then we should also do Rulej

The final product is depicted in the diagram below where we do pre-processing (FD, derived stats and hint annotations) of rewrites first and then we execute the rewrite engine.

[elsa0520]

Please assign for me and ahmad ~

[elsa0520]

Please label planner ~

[elsa0520]

Functional components issue link: #43361

[time-and-fate]

Please assign for me and ahmad ~

You can assign yourself by commenting "/assign" and assign other people by commenting "/assign @username"

[time-and-fate]

/assign @elsa0520

[elsa0520]

/assign @ghazalfamilyusa