Core: Allow customizing FileIO in REST catalog

[rdblue]

This adds setFileIOBuilder(Function<Map<String, String>, FileIO) to the RESTSessionCatalog. This allows another way to customize the FileIO instances used by the REST catalog. For example, Trino has custom FileIO implementations that manage resources differently than S3FileIO.

The function accepts a string map of config properties, which allows passing context from the catalog and table level for the IO, including token for remote S3 signing as well as the s3.* properties for access to S3.

[rdblue]

@electrum can you take a look and see if this would meet your needs?

[electrum]

I tested this out and it works for us. The simple integration where we ignore config is good enough to have everything go through TrinoFileSystem and avoid the need for HadoopFileIO:

icebergCatalogInstance.setFileIOBuilder(config -> new ForwardingFileIo(fileSystemFactory.create(identity)));

We can extend this to capture the config for overriding S3 credentials, signer, etc.

[electrum]

We'll need the same thing for JdbcCatalog.

[nastra]

LGTM once checkstyle/spotless is happy

[nastra]

nit: maybe worth inlining this, so that we don't need the null check on ioImpl:

CatalogUtil.loadFileIO(
          properties.getOrDefault(CatalogProperties.FILE_IO_IMPL, ResolvingFileIO.class.getName()),
          properties,
          conf);

[rdblue]

I updated this to be more like the JDBC catalog, where we use getOrDefault on the line above to clean it up.

[rdblue]

I looked into adding this to BaseMetastoreCatalog but it didn't make sense. That class is not Configurable and creating an IO requires a Configuration when the subclasses implement it. As a result, this class would need to override both this method and a newFileIO method so it isn't worth it.

[danielcweeks]

Seeing as how we're duplicating this API across some catalogs, should we pull the set/create into an interface?

[rdblue]

I'd like to see whether we need this more broadly first. It's a pretty narrow use case at the moment.

[danielcweeks]

+1 (pending checks)