Report error when partition schema and parquet file schema mismatch#11515
Report error when partition schema and parquet file schema mismatch#11515qqibrow wants to merge 1 commit intoprestodb:masterfrom
Conversation
I don't object merging this (helpful message is better than an unhelpuf one), but just to make sure we're on the same page -- we should be able to read those files just fine. |
|
@findepi I notice that commit hasn't updated for a while. what's the plan? shall we proceed with this commit or close it? |
|
@findepi sure. will do now. |
|
Hive's query results are different from Presto's , For exampe , The format of the table is parquet , I don't know the reason. I hope you can help me solve it. Sometimes this can appear Presto version :0211 Hive image Presto image |
qqibrow
force-pushed
the
report_error_when_file_schema_mismatch
branch
from
89819fd to
bfef553
Compare
|
error message is not related to the PR. I don't know how to re-trigger the build though. |
|
I restarted the build. |
| return columnReader.readPrimitive(field); | ||
| } | ||
| catch (UnsupportedOperationException e) { | ||
| throw new ParquetCorruptionException(format( |
There was a problem hiding this comment.
This is not really a corruption issue. Sometimes users just alter their tables and hit this issue as well.
| try { | ||
| return columnReader.readPrimitive(field); | ||
| } | ||
| catch (UnsupportedOperationException e) { |
There was a problem hiding this comment.
Instead of relying on the UnsupportedOperationException thrown by the Type classes, it may be cleaner to throw a specific exception (e.g., ParquetSchemaMismatchException or sth like that) in PrimitiveColumnReader::readValue(). What do you think?
|
This pull request has been automatically marked as stale because it has not had recent activity. If you'd still like this PR merged, please comment on the task, make sure you've addressed reviewer comments, and rebase on the latest master. Thank you for your contributions! |
5 participants
Current parquet reader will throw internal error when parquet schema and partition schema mismatch, e,g,
doublein partition schema butfloatin parquet file. This patch propose a friendly error message for that.