Move to Parquet 1.12.0

[Fokko]

Changelog:

https://github.com/apache/parquet-mr/blob/master/CHANGES.md#version-1120

[jshmchenxi]

For #2582 add support for parquet bloom filter, we need to bump parquet version

[jshmchenxi]

@kbendick Hi, you mentioned in the comment that spark versions should be concerned with parquet version upgrade. Who do you think would be appropriate to review this? The tests all passed though.

[ggershinsky]

we also need this for #2639

[kbendick]

@kbendick Hi, you mentioned in the comment that spark versions should be concerned with parquet version upgrade. Who do you think would be appropriate to review this? The tests all passed though.

@jshmchenxi - I mostly meant that it should be considered by some of the people who work closest with parquet (and also it should be ensured that it works with the more recent Spark 2.4.x and Spark 3.x versions we support).

Fortunately, @\ggershinsky is a very active Parquet committer (or possibly PMC now?) and has been doing some of the very hard work in parquet column level encryption, so if Gideon says we need this, then that's likely enough people to speak to. @\Fokko is also a parquet commiter.

Additionally, one of the PMCs / Vice President of this project, @rdblue is also a parquet PMC (in addition to a PMC or committer on what feels like every project and ), so we're in good hands in my book @jshmchenxi 👍. But thank you for considering my input. Just trying to make sure things stay sane, since it seems that some of the changes you've been putting up might not have been discussed on the dev mailing list (though I could totally have missed it). So I'm just ensuring that the right folks see potentially large changes.

[jshmchenxi]

@kbendick Sorry for the late reply, I've been busy with work. I'm new to Iceberg and should learn more. And thanks for telling me the right persons to review these!

[jshmchenxi]

@ggershinsky @Fokko @rdblue Hi, shold we be moving on to parquet 1.12.0 for iceberg now? Or maybe wait for a new version like parquet 1.12.1?

[rdblue]

This looks fine to me, but there's a conflict in versions.props and I guess 1.12.1 is out. @Fokko, can you update this?

[Fokko]

@rdblue Sure thing!

[rdblue]

Looks like there is a problem finding parquet-avro:

Execution failed for task ':iceberg-parquet:compileJava'.
> Could not resolve all files for configuration ':iceberg-parquet:compileClasspath'.
   > Could not find org.apache.parquet:parquet-avro:1.12.1.
     Searched in the following locations:
       - https://repo.maven.apache.org/maven2/org/apache/parquet/parquet-avro/1.12.1/parquet-avro-1.12.1.pom
       - https://repo.maven.apache.org/maven2/org/apache/parquet/parquet-avro/1.12.1/parquet-avro-1.12.1.jar
       - file:/home/runner/.m2/repository/org/apache/parquet/parquet-avro/1.12.1/parquet-avro-1.12.1.pom
       - file:/home/runner/.m2/repository/org/apache/parquet/parquet-avro/1.12.1/parquet-avro-1.12.1.jar
     Required by:
         project :iceberg-parquet

On maven central, it looks like 1.12.0 is the latest Parquet version. I misinterpreted @jshmchenxi's comment and though that it was released. Can you roll back to 1.12.0?

[kbendick]

@kbendick Sorry for the late reply, I've been busy with work. I'm new to Iceberg and should learn more. And thanks for telling me the right persons to review these!

No worries. Like I mentioned, this was more of a concern due to some other things I knew were going on. But it's not a concern now.

[rdblue]

Thanks, @Fokko! I merged this.