Perf: Port arrow-rs optimization for get_buffer_memory_size and add fast path for no buffer for gc string view

[zhuqi-lucas]

Which issue does this PR close?

• Closes #17007

Rationale for this change

1. Port arrow-rs optimization for get_buffer_memory_size for gc string view
2. Add fast path for no buffer case, so we don't need to do any calculation further.

What changes are included in this PR?

1. Port arrow-rs optimization for get_buffer_memory_size for gc string view
2. Add fast path for no buffer case, so we don't need to do any calculation further.

Are these changes tested?

Yes

Are there any user-facing changes?

No

[zhuqi-lucas]

FYI @alamb , i try to do some optimization in this PR:

1. Port arrow-rs optimization for get_buffer_memory_size for gc string view
2. Add fast path for no buffer case, so we don't need to do any calculation further.

Thanks!

[alamb]

Another potential idea is to simply switch to using the coalesce kernel directly (so we don't have to maintain a separate copy)

• Draft: Use upstream arrow coalesce kernel in DataFusion #16249

[alamb]

Thank you @zhuqi-lucas -- this makes sense to me

[zhuqi-lucas]

Another potential idea is to simply switch to using the coalesce kernel directly (so we don't have to maintain a separate copy)

• Draft: Use upstream arrow coalesce kernel in DataFusion #16249

Thank you @alamb for the review! I agree that switching to the upstream coalesce kernel would be the ideal follow-up solution.
Is there anything I can do to help push this forward? I'd be happy to contribute.

[alamb]

Another potential idea is to simply switch to using the coalesce kernel directly (so we don't have to maintain a separate copy)

• Draft: Use upstream arrow coalesce kernel in DataFusion #16249

Thank you @alamb for the review! I agree that switching to the upstream coalesce kernel would be the ideal follow-up solution. Is there anything I can do to help push this forward? I'd be happy to contribute.

Thanks @zhuqi-lucas -- as always super appreciated

Maybe once we upgrade to DataFusion 56 you can revive #16249 ? I think it is basically ok, but would benefit from having someone else go through the code and double check. The existing failures I think were due to other changes made at the time, and I suspect (hope) it will "just work"

[zhuqi-lucas]

Another potential idea is to simply switch to using the coalesce kernel directly (so we don't have to maintain a separate copy)

• Draft: Use upstream arrow coalesce kernel in DataFusion #16249

Thank you @alamb for the review! I agree that switching to the upstream coalesce kernel would be the ideal follow-up solution. Is there anything I can do to help push this forward? I'd be happy to contribute.

Thanks @zhuqi-lucas -- as always super appreciated

Maybe once we upgrade to DataFusion 56 you can revive #16249 ? I think it is basically ok, but would benefit from having someone else go through the code and double check. The existing failures I think were due to other changes made at the time, and I suspect (hope) it will "just work"

Thank you @alamb , i will revive it after we upgrade to 56!