[Iceberg] Support procedure remove_orphan_files#23267
[Iceberg] Support procedure remove_orphan_files#23267hantangwangd merged 1 commit intoprestodb:masterfrom
remove_orphan_files#23267Conversation
e644017 to
a9066c5
Compare
kiersten-stokes
left a comment
There was a problem hiding this comment.
This will be nice to have! Just NITs from me
presto-iceberg/src/main/java/com/facebook/presto/iceberg/procedure/RemoveOrphanFiles.java
Outdated
Show resolved
Hide resolved
presto-iceberg/src/main/java/com/facebook/presto/iceberg/procedure/RemoveOrphanFiles.java
Outdated
Show resolved
Hide resolved
a9066c5 to
4bdb47b
Compare
steveburnett
left a comment
There was a problem hiding this comment.
Thanks for the doc! Local doc build with the new table looks good. A minor suggested rephrase for active voice, and shortening for readability.
4bdb47b to
0bf44a8
Compare
steveburnett
left a comment
There was a problem hiding this comment.
LGTM! (docs)
Pull updated branch, new local docs build, looks good. Thanks!
ZacBlanco
left a comment
There was a problem hiding this comment.
A few initial comments. Great to see us getting more parity with Iceberg's Spark procedures
presto-iceberg/src/main/java/com/facebook/presto/iceberg/procedure/RemoveOrphanFiles.java
Outdated
Show resolved
Hide resolved
presto-iceberg/src/main/java/com/facebook/presto/iceberg/procedure/RemoveOrphanFiles.java
Outdated
Show resolved
Hide resolved
presto-iceberg/src/main/java/com/facebook/presto/iceberg/procedure/RemoveOrphanFiles.java
Show resolved
Hide resolved
presto-iceberg/src/main/java/com/facebook/presto/iceberg/procedure/RemoveOrphanFiles.java
Show resolved
Hide resolved
presto-iceberg/src/main/java/com/facebook/presto/iceberg/procedure/RemoveOrphanFiles.java
Outdated
Show resolved
Hide resolved
0bf44a8 to
755e834
Compare
ZacBlanco
left a comment
There was a problem hiding this comment.
Two final things. Also, I would like to see a test which exercises the default expiry limit. I don't have a good idea on how to do that in a reasonable time frame within the test, so I am happy to leave as is if you think it would be difficult
presto-iceberg/src/main/java/com/facebook/presto/iceberg/procedure/RemoveOrphanFiles.java
Outdated
Show resolved
Hide resolved
...berg/src/test/java/com/facebook/presto/iceberg/procedure/TestRemoveOrphanFilesProcedure.java
Show resolved
Hide resolved
755e834 to
be6df09
Compare
Seems it's indeed a bit difficult to test this scenario unless we change the implementation of |
Description
This PR support the procedure
remove_orphan_filesfor iceberg. It can be used to remove files which are not referenced in any metadata files of an Iceberg table and can thus be considered "orphaned".See examples as follow:
db.sampleand older than specified timestamp::db.sampleand created 3 days ago (by default)::Motivation and Context
Support removing orphan files that are not referenced in any metadata files for Iceberg
Test Plan
TestRemoveOrphanFilesProcedureContributor checklist
Release Notes