Update to arrow/parquet 53.0.0, tonic, prost, object_store, pyo3
#12032
Conversation
Co-authored-by: tison <[email protected]>
github-actions
bot
added
physical-expr
| Some(1), | ||
| Some(10), | ||
| None, | ||
| Some(0), |
There was a problem hiding this comment.
Due to apache/arrow-rs#6216 (where null counts is now Option)
alamb
changed the title
arrow/parquet `53.0.0arrow/parquet 53.0.0, tonic, prost, etc
| async-trait = "0.1.73" | ||
| aws-config = "1.5.5" | ||
| # begin pin aws-sdk crates otherwise CI MSRV check fails |
There was a problem hiding this comment.
Here is an example failure https://github.com/apache/datafusion/actions/runs/10690110122/job/29633769353?pr=12032
The same thing actually happens on main if you do cargo update in the datafusion-cli directory, just no one had hit that yet
There was a problem hiding this comment.
That is an unfortunate choice by aws.
|
This PR is now ready for review 🎉 (finally, 53.0.0 is released) |
| @@ -401,8 +401,7 @@ fn _regexp_replace_static_pattern_replace<T: OffsetSizeTrait>( | |||
| DataType::Utf8View => { | |||
| let string_view_array = as_string_view_array(&args[0])?; | |||
|
|
|||
| let mut builder = StringViewBuilder::with_capacity(string_view_array.len()) | |||
| .with_block_size(1024 * 1024 * 2); | |||
| let mut builder = StringViewBuilder::with_capacity(string_view_array.len()); | |||
There was a problem hiding this comment.
does that mean the block size is not used in the builder for this case?
There was a problem hiding this comment.
yes, we don't need it after this pr is merged: apache/arrow-rs#6136
I ran a local benchmark and show no perf diff
|
Thanks for the review @comphead and all the help @XiangpengHao -- let's get this merged! |
Draft as this is waiting on the actual arrow 53.0.0 release: apache/arrow-rs#6016Which issue does this PR close?
N/A
Rationale for this change
We need to update the DataFusion ecosystem
What changes are included in this PR?
arrow/parquet`53.0.0prost,prost-derive,pbjson,tonicecosystem #11372object_storeto 0.11.0 (TBD): Release object store0.11.0(breaking API) around Aug 15 2024 arrow-rs#6121Are these changes tested?
Yes by CI
Are there any user-facing changes?