Support S3FileIO in Hadoop and Nessie backed Iceberg tables.

[hmadison]

See Also apache/iceberg#3546.

When an non-Hive catalog is used to back Iceberg, Presto does not set up the s3, s3a, or s3n file systems for Hadoop. This prevents non-Hive catalogs from accessing files in object storage without manually passing in a hive configuration.

As a small usability improvement, I've updated IcebergResourceFactory to apply S3ConfigurationUpdater#updateConfiguration when iceberg.hadoop.config.resources is not set. This has the effect of configuring the object storage file systems for Hadoop by default, with the same configuration properties as the Hive connector.

Test plan

I ran (via Docker Compose) a test query which created a new table using the tpch.tiny.region with Nessie/MinIO backed catalog. I then queried the table back out and compared it to the source tpch.tiny.region table.

Test Resources

Docker compose file:

---
services:
  nessie:
    image: "ghcr.io/projectnessie/nessie"
    ports:
      - "19120:19120"
  minio:
    image: "quay.io/minio/minio"
    ports:
      - "9001:9001"
      - "9000:9000"
    command: |
      server /data --console-address ":9001"

Test Query:

create schema test;
create table test.region as (select * from tpch.tiny.region);

== RELEASE NOTES ==

General Changes
* Iceberg catalogues which use Hadoop or Nessie as catalogs now support reading from and writing to S3 with the same configuration options as the Hive catalog.

[tdcmeehan]

FYI the issue linked seems irrelevant

[hmadison]

FYI the issue linked seems irrelevant

My apologies, I meant to link to the Iceberg issue which discusses, in part, the error message.

[tdcmeehan]

No problem, do you want to update the commit message with apache/iceberg#3546 so it correctly links?

[agrawalreetika]

Thank you @hmadison for fixing up support for S3-backed Iceberg tables.

I have a question, Is there any reason we are not updating S3 configs always?
I mean we can also have the scenario when we have an iceberg table backed by an S3 filesystem and we want to supply some extra Hadoop config-resource in the catalog, then in that case how s3 configs would be updated?