Fix cte filter pushdown wrong results by splitting SpecialFormExpressions#22700
Conversation
fbeb8cc to
07e1101
Compare
07e1101 to
515cb5a
Compare
| } | ||
| } | ||
| return new FilterNode(node.getSourceLocation(), idAllocator.getNextId(), node, predicate); | ||
| resultPredicate = SimplifyRowExpressions.rewrite(resultPredicate, metadata, session.toConnectorSession()); |
There was a problem hiding this comment.
Simplifying the or conditions is essential and I observed a case in prod where it helped push these filters further down in predicatePushdown rule
There was a problem hiding this comment.
Curious what does this simplify do? Is it to rebalance multiple OR condition?
There was a problem hiding this comment.
Yeah looks like it breaks down the multiple ors into efficient cnf and dnf forms. In the predicate pushdown optimizer, the current visitJoin method appears to only decompose projects when they are pre broken by this optimizer.
I've noticed instances where 'or' conditions were not pushed down beneath joins.
|
Just a nit, suggest release note entry revision. |
|
The release notes should mention that it's a wrong results fix and a bit more user facing. e.g. maybe something like
|
|
Can be separate but we should add argument checks when creating special form expressions that they have the right number of arguments. |
Description
Fixes #22147. This is needed due to a bug in SpecialFormExpressions #22698 if > 2 conditions are used.
Also simplified the extra filters so that predicatePushdown optimizer could push down the conditions further
Motivation and Context
Impact
Test Plan
Contributor checklist
Release Notes
Please follow release notes guidelines and fill in the release notes below.