Convert Null to Int32(None) for MakeArray at type coercion step #7995
Conversation
Signed-off-by: jayzhan211 <[email protected]>
github-actions
bot
added
physical-expr
Signed-off-by: jayzhan211 <[email protected]>
Signed-off-by: jayzhan211 <[email protected]>
jayzhan211
force-pushed
the
default-type-for-null
branch
from
1fae14d
to
6296af1
Compare
github-actions
bot
added
logical-expr
Signed-off-by: jayzhan211 <[email protected]>
jayzhan211
changed the title
jayzhan211
changed the title
There was a problem hiding this comment.
Thank you for this contribution @jayzhan211 -- I am not quite sure how this contributes to #7142.
I apologize for the delay in reviewing
| @@ -579,26 +580,51 @@ fn coerce_arguments_for_fun( | |||
| .collect::<Result<Vec<_>>>()?; | |||
| } | |||
|
|
|||
| // Converting `Null` to Int32(None) to avoid handling `Null` in array functions | |||
There was a problem hiding this comment.
It seems very strange to me that there are three special cases for if *fun == BuiltinScalarFunction::MakeArray { that all do similar things 🤔
There was a problem hiding this comment.
I feel like the fact this is special casing NULLs when nothing else in DataFusion needs to do the same, is an indication something is not quite right with this architecture. Is there any way we can remove these special cases?
There was a problem hiding this comment.
Actually converting null is not only a special case for MakeArray. I think we need to extend this to other Array function too.
For example, array_append(make_array(1,null,3), null);
We not only need conversion in make_array but also conversion in array_append.
Which issue does this PR close?
Ref #7142.
Rationale for this change
What changes are included in this PR?
Convert null to int32(none) in type coercion, only make_array is supported now.
Are these changes tested?
Are there any user-facing changes?