Fix to avoid retrying the metadata loading once a FileNotFoundException is thrown, when trying to drop Iceberg tables, whose metadata is deleted from its S3 storage#23510
Merged
hantangwangd merged 1 commit intoprestodb:masterfrom Aug 31, 2024
Conversation
Contributor
|
Nice change @dmariamgeorge . Can you please try to create a test case for this? |
1f391ba to
258c4dc
Compare
Contributor
Author
@tdcmeehan - There exists a testcase |
258c4dc to
2545e4d
Compare
Member
hantangwangd
left a comment
There was a problem hiding this comment.
Nice catch, the change looks good to me!
Maybe in TestIcebergMetadataListing we can set iceberg.hive.table-refresh.max-retry-time to a longer period, and validate the timeOut of testTableDropWithMissingMetadata() to a shorter period, so that we can confirm that it did not trigger the retry?
2545e4d to
f36e6ea
Compare
tdcmeehan
requested changes
Aug 27, 2024
presto-iceberg/src/test/java/com/facebook/presto/iceberg/TestIcebergMetadataListing.java
Outdated
Show resolved
Hide resolved
f36e6ea to
86f0e0f
Compare
presto-iceberg/src/test/java/com/facebook/presto/iceberg/TestIcebergMetadataListing.java
Outdated
Show resolved
Hide resolved
85c35cb to
595a315
Compare
tdcmeehan
reviewed
Aug 30, 2024
presto-iceberg/src/test/java/com/facebook/presto/iceberg/TestIcebergMetadataListing.java
Show resolved
Hide resolved
595a315 to
6d4beb9
Compare
Contributor
|
Nit of formatting on the release note entry - need a space before |
6d4beb9 to
77a7330
Compare
tdcmeehan
approved these changes
Aug 30, 2024
ZacBlanco
approved these changes
Aug 30, 2024
25 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
According to the current behaviour, when a table's metadata is deleted from S3, it repeatedly tries to access the metadata file, leading to multiple retries and delays in dropping the table. with this fix, upon encountering a FileNotFoundException, it immediately halts retries and proceeds with the drop operation, completing the task in a fraction of the time.
Motivation and Context
Fixes issue: #23511
Impact
Users will experience significantly faster drop operations for Iceberg tables whose metadata has already been deleted from S3 storage. Instead of encountering long delays due to unnecessary retries, it will now recognize when the metadata is missing and complete the drop operation more quickly.
Test Plan
Contributor checklist
Release Notes