Skip to content

use correct identity to initiate delete in HDFS#13908

Merged
phd3 merged 1 commit intotrinodb:masterfrom
wliufle:delete_on_drop
Sep 8, 2022
Merged

use correct identity to initiate delete in HDFS#13908
phd3 merged 1 commit intotrinodb:masterfrom
wliufle:delete_on_drop

Conversation

@wliufle
Copy link
Copy Markdown
Contributor

@wliufle wliufle commented Aug 29, 2022

Description

Currently, ThriftHiveMetaStore uses HdfsContext to get the file system and delete the partitions when hive.metastore.thrift.delete-files-on-drop is turned on. The HdfsContext is initialized using a DEFAULT_METASTORE_USER which is presto. So when Trino initiates delete operations, user authentication uses the presto user for getting HDFS file system. This is fine when impersonation is turned off, since the underlying user can be obtained from the hadoop system in DirectHdfsAuthentication. But in ImpersonatingHdfsAuthentication, it uses the presto user to create proxy user.

Is this change a fix, improvement, new feature, refactoring, or other?
fix

Is this a change to the core query engine, a connector, client library, or the SPI interfaces? (be specific)
hdfs in hive metastore

How would you describe this change to a non-technical end user or system administrator?
Trino used to use the dummy user in ThriftHiveMetaStore to get HDFS file system and this fails when the real user is needed if it's impersonating another user.

Related issues, pull requests, and links

Documentation

(x ) No documentation is needed.
( ) Sufficient documentation is included in this PR.
( ) Documentation PR is available with #prnumber.
( ) Documentation issue #issuenumber is filed, and can be handled later.

Release notes

(x ) No release notes entries required.
( ) Release notes entries required with the following suggested text:

# Section
* Fix some things. ({issue}`issuenumber`)

Copy link
Copy Markdown
Member

@phd3 phd3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merging, as changes requested by @electrum have been addressed

@phd3 phd3 merged commit 134d14b into trinodb:master Sep 8, 2022
@github-actions github-actions bot added this to the 396 milestone Sep 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

3 participants