You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We have some processes on Glue that creates/merges into Iceberg tables. We noticed that the warehouse subfolder has created a folder with no name but a single /, screenshot attached here:
Because of this, we are facing issues (bad path due to double slash after data, like data//) while trying to read the snaphot using the python library pyiceberg (we need it as we are creating a customized tool to test the latest iceberg snapshots for data quality).
Some guidance on how to solve this or why this folder is created would be nice.
EDIT 1: The unnamed subfolder has data files in it.
EDIT 2: Some more context; I see this pattern only with tables having merge-on-read property.
The text was updated successfully, but these errors were encountered:
Are you using the HadoopFileIO or the S3FileIO? I know HadoopFileIO has some quirks with S3 where it creates psudeo-directories some times to mimic posix.
Are you using the HadoopFileIO or the S3FileIO? I know HadoopFileIO has some quirks with S3 where it creates psudeo-directories some times to mimic posix.
I am using the S3FileIO, not HadoopFileIO. Seems it is not exclusive to the file system. :(
Hello,
We have some processes on Glue that creates/merges into Iceberg tables. We noticed that the warehouse subfolder has created a folder with no name but a single /, screenshot attached here:
Because of this, we are facing issues (bad path due to double slash after data, like data//) while trying to read the snaphot using the python library pyiceberg (we need it as we are creating a customized tool to test the latest iceberg snapshots for data quality).
Some guidance on how to solve this or why this folder is created would be nice.
EDIT 1: The unnamed subfolder has data files in it.
EDIT 2: Some more context; I see this pattern only with tables having merge-on-read property.
The text was updated successfully, but these errors were encountered: