Add S3 Http Proxy configuration support

[aczajkowski]

Description

Add S3 Http Proxy configuration support for Trino

Documentation

(X) No documentation is needed.
(V) Sufficient documentation is included in this PR.
(X) Documentation PR is available with #prnumber.
(X) Documentation issue #issuenumber is filed, and can be handled later.

Release notes

(X) No release notes entries required.
(V) Release notes entries required with the following suggested text:

# Hive/Iceberg
* Support S3 Http Proxy configuration for Trino

[losipiuk]

LGTM

[losipiuk]

LGTM. Is there a way to test it? Maybe an integration test with MinIO + a HTTP proxy container (not sure if worth it).

Also did you have a chance to at least test it manually?

[aczajkowski]

LGTM. Is there a way to test it? Maybe an integration test with MinIO + a HTTP proxy container (not sure if worth it).

Also did you have a chance to at least test it manually?

@losipiuk
No did not have enough time yet to test it manually.
I was thinking on doing such Integration test with MinIO and HTTP Proxy.
But i have doubts:

• its going to be slow and memory consuming (and we are already on the edge of performance of our CI)
• seems we will be mostly testing AWS client library code.

On our site we test if configuration is properly handled. But not how TrinoS3FileSystem is configured.
Possibly i could even should add a test for that. This would cover our code well and check if config settings are properly set to S3 Client through all layers.
WDYT ??

[losipiuk]

LGTM. Is there a way to test it? Maybe an integration test with MinIO + a HTTP proxy container (not sure if worth it).
Also did you have a chance to at least test it manually?

@losipiuk No did not have enough time yet to test it manually. I was thinking on doing such Integration test with MinIO and HTTP Proxy. But i have doubts:

• its going to be slow and memory consuming (and we are already on the edge of performance of our CI)
• seems we will be mostly testing AWS client library code.

Why do you think it would be slow. It would as fast as any other integration test where we put up MinIO container. It may be a bit painful to code - but runtime-wise I do not expect it will be a problem.

On our site we test if configuration is properly handled. But not how TrinoS3FileSystem is configured. Possibly i could even should add a test for that. This would cover our code well and check if config settings are properly set to S3 Client through all layers. WDYT ??

You mean tests like we have in TestTrinoS3FileSystem. Would be nice to have.

[aczajkowski]

@losipiuk

@losipiuk No did not have enough time yet to test it manually. I was thinking on doing such Integration test with MinIO and HTTP Proxy. But i have doubts:

• its going to be slow and memory consuming (and we are already on the edge of performance of our CI)
• seems we will be mostly testing AWS client library code.

Why do you think it would be slow. It would as fast as any other integration test where we put up MinIO container. It may be a bit painful to code - but runtime-wise I do not expect it will be a problem.

From my observations they are slower. But yes it's not going to be supper heavy.
Maybe let me rephrase my doubt. I think it's going to mostly test S3 Client from AWS.
Which i believe is well tested by AWS.

On our site we test if configuration is properly handled. But not how TrinoS3FileSystem is configured. Possibly i could even should add a test for that. This would cover our code well and check if config settings are properly set to S3 Client through all layers. WDYT ??

You mean tests like we have in TestTrinoS3FileSystem. Would be nice to have.

Yes just added it :)

[aczajkowski]

@losipiuk I've adde Integration test using Apache HTTPD + MinIO to test S3 Proxy configuration.
It's on top of previous commit so previous review effects are untouched.

[aczajkowski]

@losipiuk reverted to single commit (all changes connected to Proxy Integration Test dropped)

[losipiuk]

CI: #4936

[findepi]

test coming here: #11310