[SPARK-49695][SQL][3.5] Postgres fix xor push-down #49071
Conversation
andrej-db
changed the title
MaxGekk
left a comment
MaxGekk
left a comment
There was a problem hiding this comment.
@andrej-db Please, change PR's title, and replace (3.5 backport) by the tag [3.5]. Also add a link to the original PR to PR's description.
andrej-db
changed the title
Done. @MaxGekk |
MaxGekk
changed the title
|
Also cc @LuciferYang |
|
@andrej-db The newly added test seems to have failed:
|
|
Maybe the tables don't have the same data as in |
|
Thank you @andrej-db |
| } | ||
| } | ||
|
|
||
| override def compileExpression(expr: Expression): Option[String] = { |
There was a problem hiding this comment.
Is it safe to only move this function from https://github.com/apache/spark/pull/48210/files to the branch-3.5? WDYT @MaxGekk and @yaooqinn
There was a problem hiding this comment.
#48210 is an improvement rather than a bug fix. If the current pr can solve the issue, this changes is fine to me. Of course, let's hear what others think.
There was a problem hiding this comment.
Let's backport only fix related code.
|
also cc @dongjoon-hyun |
### What changes were proposed in this pull request? Backport of the #48144 This PR fixes the pushdown of ^ operator (XOR operator) for Postgres. Those two databases use this as exponent, rather then bitwise xor. Fix is consisted of overriding the SQLExpressionBuilder to replace the '^' character with '#'. ### Why are the changes needed? Result is incorrect. ### Does this PR introduce _any_ user-facing change? Yes. The user will now have a proper translation of the ^ operator. ### How was this patch tested? ### Was this patch authored or co-authored using generative AI tooling? No. Closes #49071 from andrej-db/PGXORBackport. Lead-authored-by: Andrej Gobeljić <[email protected]> Co-authored-by: andrej-gobeljic_data <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
|
Merged to branch-3.5 for Apache Spark 3.5.4. |
5 participants
What changes were proposed in this pull request?
Backport of the #48144
This PR fixes the pushdown of ^ operator (XOR operator) for Postgres. Those two databases use this as exponent, rather then bitwise xor.
Fix is consisted of overriding the SQLExpressionBuilder to replace the '^' character with '#'.
Why are the changes needed?
Result is incorrect.
Does this PR introduce any user-facing change?
Yes. The user will now have a proper translation of the ^ operator.
How was this patch tested?
Was this patch authored or co-authored using generative AI tooling?
No.