Support List type coercion for CASE-WHEN-THEN expression #12490
Conversation
github-actions
bot
added
optimizer
|
I tried to add some sql test for |
goldmedal
changed the title
There was a problem hiding this comment.
Thank you @goldmedal -- this is also really nicely implemented, documented, and tested 🏅
I don't have anything to add here
cc @jayzhan211
| macro_rules! test_case_expression { | ||
| ($expr:expr, $when_then:expr, $case_when_type:expr, $then_else_type:expr, $schema:expr) => { | ||
| let case = Case { | ||
| expr: Some(Box::new(col($expr))), | ||
| when_then_expr: $when_then, | ||
| else_expr: None, | ||
| }; | ||
|
|
||
| let case_when_common_type = $case_when_type; | ||
| let then_else_common_type = $then_else_type; | ||
| let expected = cast_helper( | ||
| case.clone(), | ||
| &case_when_common_type, | ||
| &then_else_common_type, | ||
| &$schema, | ||
| ); | ||
|
|
||
| let actual = coerce_case_expression(case, &$schema)?; | ||
| assert_eq!(expected, actual); | ||
| }; | ||
| } |
There was a problem hiding this comment.
It seems two test_case_expression are the same, could we only keep one?
There was a problem hiding this comment.
Sounds great. I tried to simplify them in a8ea6cf
The position is incorrect, |
| // Coerce to the left side FixedSizeList type if the list lengths are the same, | ||
| // otherwise coerce to list with the left type for dynamic length | ||
| (FixedSizeList(lf, ls), FixedSizeList(_, rs)) => { | ||
| if ls == rs { | ||
| Some(lhs_type.clone()) | ||
| } else { | ||
| Some(List(Arc::clone(lf))) | ||
| } | ||
| } |
There was a problem hiding this comment.
Because we can't cast a FixedSizeList to a different length FixedSizeList,
> select arrow_cast(arrow_cast([1,2], 'FixedSizeList(2, Int64)'), 'FixedSizeList(3, Int64)');
This feature is not implemented: Unsupported CAST from FixedSizeList(Field { name: "item", data_type: Int64, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }, 2) to FixedSizeList(Field { name: "item", data_type: Int64, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }, 3)
I choose to use the List for the dynamic length.
There was a problem hiding this comment.
That makes sense to me - the alternative is to just reject the query -- I think this is reasonable
Thanks @jayzhan211 for solving my question! It was very helpful to me. I added more tests for the Could @alamb and @Weijun-H also double-check this change? Many thanks 🙇 |
| // Coerce to the left side FixedSizeList type if the list lengths are the same, | ||
| // otherwise coerce to list with the left type for dynamic length | ||
| (FixedSizeList(lf, ls), FixedSizeList(_, rs)) => { | ||
| if ls == rs { | ||
| Some(lhs_type.clone()) | ||
| } else { | ||
| Some(List(Arc::clone(lf))) | ||
| } | ||
| } |
There was a problem hiding this comment.
That makes sense to me - the alternative is to just reject the query -- I think this is reasonable
|
ship it! |
|
🚀 |
* support list type coercion * add planing and sql tests * clippy * support to compare nested type for case-when expression * simplify the macro rules * fix the FixedSizeList type coercion and add tests * add test for THEN-ELSE
Which issue does this PR close?
Closes #12370.
Rationale for this change
The rule for the List type is to choose the wider type (LargeList > List > FixedSizeList) as the target type.
What changes are included in this PR?
thenexpressionscase-whenexpressionAre these changes tested?
yes
Are there any user-facing changes?
no