Enable auto purge default catalog property#11749
Merged
findepi merged 1 commit intotrinodb:masterfrom May 5, 2022
Merged
Conversation
Contributor
Author
posulliv
approved these changes
Apr 19, 2022
Contributor
posulliv
left a comment
posulliv
left a comment
There was a problem hiding this comment.
This looks good to me. I just had some minor comments. Thanks for working on this!
@hashhar @findepi this change will help a lot with issues we are experiencing where DROP TABLE operations are timing out on the metastore side when dropping tables on S3 storage because it is so inefficient to copy a table to the .Trash folder.
plugin/trino-hive/src/test/java/io/trino/plugin/hive/TestHiveConfig.java
Outdated
Show resolved
Hide resolved
findepi
changed the title
findepi
reviewed
Apr 20, 2022
plugin/trino-hive/src/test/java/io/trino/plugin/hive/TestHiveConfig.java
Outdated
Show resolved
Hide resolved
plugin/trino-hive/src/main/java/io/trino/plugin/hive/HiveConfig.java
Outdated
Show resolved
Hide resolved
sravanip09
force-pushed
the
11575_auto_purge_default
branch
from
42dbce4 to
cda38b5
Compare
Contributor
Author
Contributor
Author
mosabua
requested changes
Apr 29, 2022
sravanip09
force-pushed
the
11575_auto_purge_default
branch
from
f4134bc to
ebf57fc
Compare
Contributor
Author
findepi
reviewed
May 4, 2022
mosabua
reviewed
May 4, 2022
Contributor
Author
findepi
force-pushed
the
11575_auto_purge_default
branch
from
e1de9cb to
ec32b6d
Compare
Member
|
(squashed and rebased) |
Add config property for the existing `auto_purge` table property.
findepi
force-pushed
the
11575_auto_purge_default
branch
from
ec32b6d to
ebd116c
Compare
findepi
approved these changes
May 5, 2022
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This is the continuation of effort done as part of #9457
Enabled hive.managed-table-auto-purge-default optional property at catalog level & will be enforced if auto_purge at table level is not set. This serves as default value for table property auto_purge & removes the
need to set auto_purge each and every time managed table is created. As mentioned in #11575 user is also seeing issue trino:> drop table withacid;
Query failed: The following metastore delete operations failed: drop table with acid
io.trino.spi.TrinoException: The following metastore delete operations failed: drop table withacid
This is happening as auto_purge is not set to true. hive.managed-table-auto-purge-default optional parameter gives flexibility to admin to enforce this at catalog level
there by apply to all the tables that will be created.
This change is a improvement.
This is a change to Hive connector
auto_purge = true allows to skip trash & speed up the dropping managed tables for hive connector. Current implementation requires user to specify the flag during the table creation. With this change
it gives the flexibility to set the auto_purge at catalog level.
Related issues, pull requests, and links
Related Issue: #11575
Related PR: #9457
Documentation
( ) No documentation is needed.
(x) Sufficient documentation is included in this PR.
() Documentation PR is available with #prnumber.
( ) Documentation issue #issuenumber is filed, and can be handled later.
Release notes
( x) No release notes entries required.
() Release notes entries required with the following suggested text: