[native] Add positional deletes to Iceberg native tpcds e2e tests#22867
Conversation
|
@yingsu00 Please review. |
...java/com/facebook/presto/nativeworker/TestPrestoNativeIcebergTpcdsWithPositionalDeletes.java
Outdated
Show resolved
Hide resolved
...java/com/facebook/presto/nativeworker/TestPrestoNativeIcebergTpcdsWithPositionalDeletes.java
Outdated
Show resolved
Hide resolved
...java/com/facebook/presto/nativeworker/TestPrestoNativeIcebergTpcdsWithPositionalDeletes.java
Outdated
Show resolved
Hide resolved
...java/com/facebook/presto/nativeworker/TestPrestoNativeIcebergTpcdsWithPositionalDeletes.java
Outdated
Show resolved
Hide resolved
...java/com/facebook/presto/nativeworker/TestPrestoNativeIcebergTpcdsWithPositionalDeletes.java
Outdated
Show resolved
Hide resolved
...java/com/facebook/presto/nativeworker/TestPrestoNativeIcebergTpcdsWithPositionalDeletes.java
Outdated
Show resolved
Hide resolved
...java/com/facebook/presto/nativeworker/TestPrestoNativeIcebergTpcdsWithPositionalDeletes.java
Outdated
Show resolved
Hide resolved
...java/com/facebook/presto/nativeworker/TestPrestoNativeIcebergTpcdsWithPositionalDeletes.java
Outdated
Show resolved
Hide resolved
30aa3b9 to
b3a6e4a
Compare
| Map<String, Long> deletedRowsMap = new HashMap<>(); | ||
|
|
||
| @Override | ||
| protected void doDeletes() |
There was a problem hiding this comment.
These delete queries are the same for positional delete and equality delete, and . I think they better be in the parent class TestPrestoNativeIcebergTpcdsQueriesParquetUsingThrift.
There was a problem hiding this comment.
Presto doesn't support writing equality delete yet, so it's ok to remove this class after moving all common queries to the parent class. In the future if equality delete yet is supported, then we can subclass TestPrestoNativeIcebergTpcdsQueriesParquetUsingThrift to TestPrestoNativeIcebergTpcdsQueriesParquetUsingThriftWithPositionalDelete and TestPrestoNativeIcebergTpcdsQueriesParquetUsingThriftWithEqualityDelete, and these two classes should be only a few lines which is used to set the delete method.
There was a problem hiding this comment.
We are doing the equality deletes using the Iceberg API, since there is no support to enforce equality deletes using DELETE. So this function will have a different implementation for equality delete test case.
There was a problem hiding this comment.
I will move it now to base class. If there is support to enforce the type of delete in the future, we can keep it here. Or else, we will move it back to derived classes.
|
|
||
| protected void dropTables() | ||
| { | ||
| for (String table : tpcdsFactTableNames) { |
There was a problem hiding this comment.
Why is the implementation in this class? Can any of AbstractTestNativeTpcdsQueries's children have different implementation than this? If not, let's move it to AbstractTestNativeTpcdsQueries's
There was a problem hiding this comment.
Will move it to base class.
b3a6e4a to
7899740
Compare
|
@yingsu00 addressed the comments here. Please check. |
|
@nmahadevuni Looks like this PR has introduced a test failure which is blocking PR merges. Can you please take a look. See https://app.circleci.com/pipelines/github/prestodb/presto/17633/workflows/314944a1-d3c2-4aba-a170-b3fd171d9cff/jobs/69606 |
|
@ajaygeorge, Looks like the test started failing with this #23138 merge. |
Description
Added delete phase to the Iceberg tpc-ds native e2e tests.
Motivation and Context
Goal to add full data maintenance phases described in the tpc-ds spec to the e2e tests.
Impact
No impact
Test Plan
Added a new test to run deletes, verify and run all the tpc-ds queries.