Support reading Iceberg S3 paths with double slashes#11998
Support reading Iceberg S3 paths with double slashes#11998electrum merged 1 commit intotrinodb:masterfrom
Conversation
|
@rdblue this fixes the issue you encountered |
|
What problem is it solving?
thanks for context. What if the normalized path was used later? it should just work, shouldn't it? cc @homar as it may affect vacuuming logic. |
|
This has nothing to do with delete deltas. The issue is that Glue writes S3 key names that contain double slashes, which is perfectly legal for an S3 key, but cannot be transported through an HDFS path. |
|
I don't know a good way to test this, since we can't write such paths with Trino. I don't feel that it's worth the effort to try to construct such a table manually for a test. |
thanks for clarifying
Sounds like it may or may not be related. Deletion deltas have paths included, so if something was replacing repeated slashes (eg because of going via some intermediate representation), the deletion deltas wouldn't apply. |
Description
Allow reading Iceberg tables written by Glue that have locations containing double slashes. This type of path is not possible to represent by the Hadoop
Pathobject since it normalizes paths, so we can hide the original path in the URI fragment.Related issues, pull requests, and links
Fixes #11964
Documentation
(x) No documentation is needed.
Release notes
(x) Release notes entries required with the following suggested text: