Support extracting Int8
, Int16
, Int32
statistics from Parquet Data Pages
#10928
Labels
enhancement
New feature or request
Is your feature request related to a problem or challenge?
Part of #10922
We are adding APIs to efficiently convert the data stored in Parquet's "PageIndex" into
ArrayRef
s -- which will make it significiantly easier to use this information for pruning and other tasks.Describe the solution you'd like
Add support to
StatisticsConverter::min_page_statistics
andStatisticsConverter::max_page_statistics
for the types abovedatafusion/datafusion/core/src/datasource/physical_plan/parquet/statistics.rs
Lines 637 to 656 in 2f43476
Describe alternatives you've considered
test_int64
datafusion/datafusion/core/tests/parquet/arrow_statistics.rs
Line 475 in 2f43476
arrow_statistics
tests #10927)datafusion/datafusion/core/src/datasource/physical_plan/parquet/statistics.rs
Lines 575 to 586 in 2f43476
datafusion/datafusion/core/src/datasource/physical_plan/parquet/statistics.rs
Line 90 in 2f43476
Additional context
No response
The text was updated successfully, but these errors were encountered: