-
Notifications
You must be signed in to change notification settings - Fork 737
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support casting between BinaryView <--> Utf8 and LargeUtf8 #6180
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you @xinlifoobar -- I think this code does what the ticket described
However, I think this implementation's speed could be improved significantly.
Any chance you are also willing to add a benchmark in this (or another PR)? Perhaps in
arrow-rs/arrow/benches/cast_kernels.rs
Line 153 in 01407f4
let string_view_array = cast(&dict_array, &DataType::Utf8View).unwrap(); |
I think important benchmarks would be:
Utf8Array --> StringViewArray
Utf8Array --> BinaryViewArray
BinaryViewArray --> Utf8Array
StringViewArray --> Utf8Array
StringViewArray --> BinaryViewArray
BinaryViewArray --> StringViewArray
Done |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you @xinlifoobar and @XiangpengHao -- this is great.
I went over this PR carefully and left some code suggestions but I also think they could be done as a follow on PR. I'll plan to merge this PR tomorrow to give a chance to respond.
Thanks again
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @xinlifoobar -- this looks really nice
🚀 |
Which issue does this PR close?
Closes #6162 and related #6163, apache/datafusion#11752
Rationale for this change
What changes are included in this PR?
Are there any user-facing changes?