Fix file location in case of '/' root#17808
Conversation
I can change |
| * A slash will be added between the current path and the new path element if needed. | ||
| * | ||
| * @throws IllegalArgumentException if the new path element is empty or starts with a slash | ||
| * @throws IllegalArgumentException if the new path element is empty |
There was a problem hiding this comment.
I don't think we should be changing Location.appendPath API.
This can unwanted consequences. More conservative approach would be to keep disallowing leading / here and fix listing instead.
| int index = root.endsWith("/") ? root.length() : root.length() + 1; | ||
| Location location = listingLocation.appendPath(path.substring(index)); |
|
I don't think unit testing is sufficient for this kind of problem. |
|
Created #17848 to test reading from Hive partitioned/unpartitioned table having the location on top of the S3 bucket. |
|
closing then |
4 participants
Description
Before this fix, when
rootwas/, the first file name's character was trimmed.For example when
listingLocation="s3a://bucket/",root = "/",path = "/file.parquet"then location turned out to bes3a://bucket/ile.parquet.Release notes
( ) This is not user-visible or docs only and no release notes are required.
( ) Release notes are required, please propose a release note for me.
( X ) Release notes are required, with the following suggested text: