Fix Iceberg dereference pushdown when column has a comment#11104
Fix Iceberg dereference pushdown when column has a comment#11104findepi merged 1 commit intotrinodb:masterfrom
Conversation
plugin/trino-iceberg/src/test/java/io/trino/plugin/iceberg/BaseIcebergConnectorTest.java
Outdated
Show resolved
Hide resolved
|
Good catch, thanks! |
c808db2 to
f4a1146
Compare
alexjo2144
left a comment
There was a problem hiding this comment.
@findepi mind taking a look?
plugin/trino-iceberg/src/main/java/io/trino/plugin/iceberg/IcebergMetadata.java
Outdated
Show resolved
Hide resolved
|
@BlueStalker @alexjo2144 the fix is quite indirect. What's the root cause of the failure? |
The original code (applyProject), put an new Assignment with column (without comment) into ProjectionApplicationResult, and I assume later in will be available in TableScan Node, When do 'applyFilter' from PushPredicateIntoTableScan, it will contain duplicate (IcebergColumnHandle) one with comment and the other without, which will finally lead to exception of (Multiple entries with same key: .... ), also @phd3 said if applyProjection can just bypass the non nested column, it should also solve the problem. |
f4a1146 to
a60ddfd
Compare
plugin/trino-iceberg/src/test/java/io/trino/plugin/iceberg/BaseIcebergConnectorTest.java
Outdated
Show resolved
Hide resolved
plugin/trino-iceberg/src/test/java/io/trino/plugin/iceberg/BaseIcebergConnectorTest.java
Outdated
Show resolved
Hide resolved
plugin/trino-iceberg/src/main/java/io/trino/plugin/iceberg/IcebergMetadata.java
Outdated
Show resolved
Hide resolved
plugin/trino-iceberg/src/main/java/io/trino/plugin/iceberg/IcebergMetadata.java
Outdated
Show resolved
Hide resolved
a60ddfd to
a8eaa09
Compare
|
@findepi checked it again? |
plugin/trino-iceberg/src/main/java/io/trino/plugin/iceberg/IcebergMetadata.java
Outdated
Show resolved
Hide resolved
a8eaa09 to
41f6761
Compare
|
making a couple more small changes here. |
|
pushed commit title change (and renamed the PR) |
41f6761 to
15548ab
Compare
15548ab to
4066e2b
Compare
When Query iceberg table with predicate column including comments, and deference pushdown enabled. Exception occurred if the predicates containing both nested column and non_nested, for example
select count(*) from table where col1 = 1 and col2.n1 = 3will show exception like
Also refer the testing cases.