[SPARK-17075][SQL] Follow up: fix file line ending and improve the tests #17051
Conversation
|
Test build #73401 has started for PR 17051 at commit |
| } | ||
|
|
||
| } | ||
| /* |
There was a problem hiding this comment.
please make sure the only change is line ending
There was a problem hiding this comment.
The diff shown here is because I also removed an unused import. I'll revert it to make review easier.
There was a problem hiding this comment.
Done in bc1d2a6 . But that seems not affecting the diffs.
|
Test build #73404 has started for PR 17051 at commit |
| // AND/OR/NOT), swap the sides of the attribte and the literal, reverse the | ||
| // operator, and then check again. | ||
| val rewrittenFilter = filterNode transformExpressionsDown { | ||
| case op @ EqualTo(ar: AttributeReference, l: Literal) => |
There was a problem hiding this comment.
nit: case b @ BinaryComparison(ar: AttributeReference, l: Literal) => b.withNewChildren(l, ar)
There was a problem hiding this comment.
👍
I tried to find something like this but failed to, so I resorted to the current code. Thanks for the tip!
There was a problem hiding this comment.
Emm, we not only switch the side of the attr and the literal, but also reverse the operator, e.g. LessThan would be changed to GreaterThan, so we can keep the filter output not affected. So I guess we can't use withNewChildren here.
|
jenkins retest this please |
|
Test build #73414 has finished for PR 17051 at commit
|
|
thanks, merging to master! |
## What changes were proposed in this pull request? Fixed the line ending of `FilterEstimation.scala` (It's still using `\n\r`). Also improved the tests to cover the cases where the literals are on the left side of a binary operator. ## How was this patch tested? Existing unit tests. Author: Shuai Lin <[email protected]> Closes apache#17051 from lins05/fix-cbo-filter-file-encoding.
3 participants
What changes were proposed in this pull request?
Fixed the line ending of
FilterEstimation.scala(It's still using\n\r). Also improved the tests to cover the cases where the literals are on the left side of a binary operator.How was this patch tested?
Existing unit tests.