Add option to force S3 region to connect to

[skrzypo987]

Description

Pretty straightforward change allowing to force S3 client to use specified region

Non-technical explanation

Add option to force specific region when connecting to S3

Release notes

( ) This is not user-visible or docs only and no release notes are required.
( ) Release notes are required, please propose a release note for me.
(x) Release notes are required, with the following suggested text:

# Section
* Add option to force specific region when connecting to S3

[hashhar]

@findepi / @ebyhr Reminder to make sure Glue and cloud-only tests are run before merge.

[skrzypo987]

@findepi / @ebyhr Reminder to make sure Glue and cloud-only tests are run before merge.

@hashhar Do I need to add some tags to enable them?

[hashhar]

a non-fork PR needs sending since it requires secrets. Unfortunately nothing you can do for it yet @skrzypo987.

[ebyhr]

Can we update hive.rst?

[findepi]

Test PR with secrets: #14419
(@nineinchnick's #12817 will make these PRs easier)

[alexjo2144]

You should use the region even if endpoint isn't set

        String endpoint = hadoopConfig.get(S3_ENDPOINT);
        String region = hadoopConfig.get(S3_REGION);
        if (endpoint != null) {
            clientBuilder.setEndpointConfiguration(new EndpointConfiguration(endpoint, region));
            regionOrEndpointSet = true;
        }
        else if (region != null) {
            clientBuilder.setRegion(region);
            regionOrEndpointSet = true;
        }

[skrzypo987]

You are actually right. I totally ignored the fact that setting regionOrEndpointSet to true will prevent US_EAST_1 to be set as a default few lines later.
It will not effectively change anything as US_EAST_1 is always chosen as a default one, but we should set it nonetheless.

Made the change and altered tests.

[alexjo2144]

👍

[colebow]

If we're going to add a configuration property, we also need to document it. I'll submit a follow-up PR.

cc @Praveen2112