Upgrade Parquet to version 1.12.2#54
Conversation
e80649a to
5ace87e
Compare
|
@aweisberg, can we build a staging jar to have a try in presto repo, like we did when upgrading to 1.11.x? Thanks in advance! |
|
I deployed hive-apache-3.0.0-8-20220307.153144-32 Thanks! |
|
Thanks @aweisberg! I created PR with this version but found building errors because of the zstd-jni duplicate but version mismatch. So I just add a fix here to exclude zstd-jni in parquet. Can you help to release another new version to try out? |
|
Deployed 3.0.0-8-20220309.171836-34 |
|
Thanks. This new version succeeded (PR). Can you merge and release an official version? |
aweisberg
left a comment
There was a problem hiding this comment.
OK, LGTM. Will release.
|
I just released 3.0.0-8, it should appear in mirrors soon. |
|
@shangxinli Do we really need to upgrade Avro? After upgrading avro to 1.10.1, Presto fails to query Hudi MOR tables. The reason is that there is a breaking change in GenericData class Old version: @Override
public Object get(String key) {
Field field = schema.getField(key);
if (field == null)
return null;
return values[field.pos()];
}New version: @Override
public Object get(String key) {
Field field = schema.getField(key);
if (field == null) {
throw new AvroRuntimeException("Not a valid schema field: " + key);
}
return values[field.pos()];
}ref: prestodb/presto#17463 |
|
In that case, we can upgrade avro in Hudi to support that? I guess we need to upgrade sooner or later. |
No description provided.