Skip to content

Improve listing performance of Hudi tables#17084

Merged
arunthirupathi merged 1 commit intoprestodb:masterfrom
codope:hudi-metadata-listing
Jan 4, 2022
Merged

Improve listing performance of Hudi tables#17084
arunthirupathi merged 1 commit intoprestodb:masterfrom
codope:hudi-metadata-listing

Conversation

@codope
Copy link
Contributor

@codope codope commented Dec 9, 2021

  • Integrate metadata-based listing for Hudi tables. This is
    enabled by a session property.
  • Implement a new DirectoryLister for Hudi that uses
    HoodieTableFileSystemView to fetch data files.
  • Bump Hudi version to 0.10.0 to use above features.

Test plan -
Synced a Hudi table to Hive and queried through Presto
with both metadata enabled and disabled.

== NO RELEASE NOTE ==

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants