Fixes for file renaming and manifest features#15536
Conversation
b886ccf to
ee36cea
Compare
presto-hive/src/main/java/com/facebook/presto/hive/StoragePartitionLoader.java
Outdated
Show resolved
Hide resolved
presto-hive/src/main/java/com/facebook/presto/hive/HiveManifestUtils.java
Outdated
Show resolved
Hide resolved
presto-hive/src/main/java/com/facebook/presto/hive/HiveManifestUtils.java
Outdated
Show resolved
Hide resolved
presto-hive/src/main/java/com/facebook/presto/hive/HiveManifestUtils.java
Outdated
Show resolved
Hide resolved
presto-hive/src/main/java/com/facebook/presto/hive/HiveManifestUtils.java
Outdated
Show resolved
Hide resolved
If we append 0s for bucketed tables, why sorting will be a problem? Or even with bucketed tables + file renaming, we will not do 0 appending anymore? |
In normal scenario, for bucketed tables the names are like "file_prefix + padded_bucket_number" . In file renaming case, we dont append 0 at the beginning of the filename because we dont know how many files are there in the partition . Hence we dont know how many 0s we should pad. |
depended by https://github.com/facebookexternal/presto-facebook/pull/1363