Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

simplify_expressions return expression with wrong type. #6596

Open
jackwener opened this issue Jun 8, 2023 · 4 comments
Open

simplify_expressions return expression with wrong type. #6596

jackwener opened this issue Jun 8, 2023 · 4 comments
Assignees
Labels
bug Something isn't working

Comments

@jackwener
Copy link
Member

jackwener commented Jun 8, 2023

Describe the bug

Some expression after simplify_expressions return different type like.

Internal error: Optimizer rule 'simplify_expressions' failed, due to generate a different schema, original schema: DFSchema { fields: [DFField { qualifier: None, field: Field { name: "array_fill(Int64(11),make_array(Int64(1),Int64(2),Int64(3)))", data_type: List(Field { name: "item", data_type: Int64, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }), nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} } }, DFField { qualifier: None, field: Field { name: "array_fill(Int64(3),make_array(Int64(2),Int64(3)))", data_type: List(Field { name: "item", data_type: Int64, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }), nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} } }, DFField { qualifier: None, field: Field { name: "array_fill(Int64(2),make_array(Int64(2)))", data_type: List(Field { name: "item", data_type: Int64, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }), nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} } }], metadata: {} }, new schema: DFSchema { fields: [DFField { qualifier: None, field: Field { name: "array_fill(Int64(11),make_array(Int64(1),Int64(2),Int64(3)))", data_type: List(Field { name: "item", data_type: List(Field { name: "item", data_type: List(Field { name: "item", data_type: Int64, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }), nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }), nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }), nullable: false, dict_id: 0, dict_is_ordered: false, metadata: {} } }, DFField { qualifier: None, field: Field { name: "array_fill(Int64(3),make_array(Int64(2),Int64(3)))", data_type: List(Field { name: "item", data_type: List(Field { name: "item", data_type: Int64, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }), nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }), nullable: false, dict_id: 0, dict_is_ordered: false, metadata: {} } }, DFField { qualifier: None, field: Field { name: "array_fill(Int64(2),make_array(Int64(2)))", data_type: List(Field { name: "item", data_type: Int64, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }), nullable: false, dict_id: 0, dict_is_ordered: false, metadata: {} } }], metadata: {} }. This was likely caused by a bug in DataFusion's code and we would welcome that you file an bug report in our issue tracker
[SQL] select array_fill(11, make_array(1, 2, 3)), array_fill(3, make_array(2, 3)), array_fill(2, make_array(2));

To Reproduce

#6595

cargo test -p datafusion --test sqllogictests

Expected behavior

No response

Additional context

No response

@jackwener jackwener added the bug Something isn't working label Jun 8, 2023
@jackwener jackwener changed the title Some expression return wrong expression Some ScalarFunction expression return wrong expression Jun 8, 2023
@jackwener jackwener changed the title Some ScalarFunction expression return wrong expression simplify_expressions return expression with wrong type. Jun 8, 2023
@jackwener
Copy link
Member Author

SELECT DATE_TRUNC('MINUTE', TIMESTAMP '2022-08-03 14:38:50Z');

"Projection: date_trunc(Utf8("MINUTE"), CAST(Utf8("2022-08-03 14:38:50Z") AS Timestamp(Nanosecond, None)))
  EmptyRelation"
-->
"Projection: TimestampSecond(1659537480, None) AS date_trunc(Utf8("MINUTE"),Utf8("2022-08-03 14:38:50Z"))
  EmptyRelation"

expr type from timestamp nanosecond -> timestamp second.

@jackwener
Copy link
Member Author

jackwener commented Jun 8, 2023

I already fix bug about type conversion like aboving.

We need investigate the remaining problem related with simplify expression about arrays

@izveigor
Copy link
Contributor

izveigor commented Jun 8, 2023

@jackwener As I understand the main problem is in the nullable parameter.
I think the solution to the problem is #6556

@alamb
Copy link
Contributor

alamb commented Jun 13, 2023

FWIW the date_trunc issue from #6596 (comment) is tracked in #6623

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants