Properly initialize Iceberg REST catalog Configuration

[alexjo2144]

Description

Without these configurationInitializers catalog properties are not properly sent to the file system implementation.

Additional context and related issues

#13294

Release notes

( ) This is not user-visible or docs only and no release notes are required.
(x) Release notes are required, please propose a release note for me.
( ) Release notes are required, with the following suggested text:

[electrum]

Why do we need to give it Hadoop properties? The setConf() calls are removed entirely here: #16454

[alexjo2144]

We either need to set it or change the FileIO implementation that's being used. Because we delegate to the core Iceberg library for the RESTTableOperations, the FileIO that gives us back right now is a HadoopFileIO which needs the Configuration.

In the other TrinoCatalog implementations we use ForwardingFileIO but that's not plumbed in here yet.

[electrum]

We should fix it to use ForwardingFileIO.

[electrum]

I looked through the code and there doesn't seem to be an obvious way to add it here. We might be able to do it by subclassing RESTSessionCatalog and making tableFileIO() protected, but that will require a new release of Iceberg.

For now, we might need to fork RESTSessionCatalog. It would be good for us to implement RESTClient using Airlift HTTP client so that we get the standard functionality and configuration.

TrinoJdbcCatalog using JdbcCatalog has the same problem. This seems fixable by subclassing JdbcCatalog and overriding newTableOps().

[alexjo2144]

@electrum how would you feel about merging this fix for now and getting ForwardingFileIO into here and JDBC after? The rest catalog is completely broken as is.

[danielcweeks]

@alexjo2144 You can set the Catalog FileIO via the RESTSessionCatalog initialization here

properties.put(CatalogProperties.FILE_IO_IMPL, ForwardingFileIO.class.getName())

[alexjo2144]

Thanks Dan, I saw that but the ForwardingFileIO takes a TrinoFileSystem argument in the constructor and I couldn't figure out how to get that in there.

[alexjo2144]

I see, we can use ForwardingFileIO today if we implement HadoopConfigurable

[electrum]

@dain and I had a call with @rdblue and @danielcweeks this morning. Ryan is going to propose an API change Iceberg 1.2.0 for the catalogs that lets us provide a FileIO for a given configuration.

[electrum]

We'll be able to fix this with Iceberg 1.2.0 thanks to apache/iceberg#7064

[alexjo2144]

Thanks David, if you'd rather wait for that fix in 1.2.0 I'll go ahead and close this