Support writing Parquet encoding stats#9569
Conversation
b4478e9 to
fd0c063
Compare
lib/trino-parquet/src/main/java/io/trino/parquet/writer/PrimitiveColumnWriter.java
Outdated
Show resolved
Hide resolved
lib/trino-parquet/src/main/java/io/trino/parquet/writer/PrimitiveColumnWriter.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
how small can the smallest page be? can we overflow here?
There was a problem hiding this comment.
The field this in PageEncodingStats this gets used for is also an int, so it should be fine.
lib/trino-parquet/src/main/java/io/trino/parquet/writer/PrimitiveColumnWriter.java
Outdated
Show resolved
Hide resolved
fd0c063 to
313923b
Compare
|
For testing, I can add a test with something like: And assert that |
better such test than nothing, imo |
Encoding stats are used by the reader to check if the dictionary pages can be used for predicate pushdown.
313923b to
6b958f7
Compare
Added |
|
|
|
CI #9617 |
Encoding stats are used by the reader to check if the dictionary pages can be used for predicate pushdown.
Fixes #9554
TODO: Still need to add a test, but I did manually test that there's dictionary based pushdown happening now that didn't happen before.