AWS: support S3FileIO alternative endpoint and credentials

[jackye1995]

As discussed in the mailing list: https://mail-archives.apache.org/mod_mbox/iceberg-dev/202112.mbox/%3CCAMwmD1_P2PCMpzUai5RV%2B4a9Bv62ZFqrrDdxCtLcTSvPnUxk6g%40mail.gmail.com%3E

For DefaultClientFactory, we allow configuration of S3 static credentials and endpoint.
For AssumeRoleClientFactory, we only allow configuration of S3 endpoint, because credential is always assume role credential.

@rdblue @danielcweeks @findepi @mayursrivastava

[jackye1995]

This is a bit hard to test, I will try to add integration tests, meanwhile @findepi @mayursrivastava could you check if the added configurations are enough for using S3FileIO in GCS, MinIO and other services?

[jackye1995]

tests added, and I've run AWS integration tests to verify all tests pass.

[kbendick]

Thanks for adding this @jackye1995!

[rdblue]

Looks good to me, but I'm not very familiar with this part. It would be great if @danielcweeks could take a look, too.

[danielcweeks]

This all looks good to me. But for those looking to use endpoint override for GCS, I would point you to GCSFileIO #3711

[jackye1995]

Thanks for the review Daniel!

For anyone looking into this, we are adding these configs for users who want to use S3 compatible storage services and frameworks such as MinIO through the AWS S3 client. But ultimately the right way to go is to have a native FileIO, like in the case of GCS, which can unlock all the features of a storage service.