Add S3 connection ttl configuration to reduce http server exception

[hackeryang]

Description

Add Amazon S3 connection ttl configuration to reduce http server errors in high load scenarios.

Additional context and related issues

Fixes #16005

Release notes

( ) This is not user-visible or docs only and no release notes are required.
(x) Release notes are required, please propose a release note for me.
( ) Release notes are required, with the following suggested text:

# Section
* Add Amazon S3 connection TTL configuration to reduce HTTP server errors in high-load scenarios. ([{issue}`#16005`](https://github.com/trinodb/trino/issues/16005))

[hackeryang]

Hi @sopel39 @alexjo2144 , can you please take a look when you have time, thanks~

[sopel39]

@findepi please take over this one

[findepi]

@hackeryang can you please fix the build errors?
You can use https://gist.github.com/findepi/04c96f0f60dcc95329f569bb0c44a0cd#run-static-code-analysis to run the checkstyle & error-prone locally

[hackeryang]

@hackeryang can you please fix the build errors? You can use https://gist.github.com/findepi/04c96f0f60dcc95329f569bb0c44a0cd#run-static-code-analysis to run the checkstyle & error-prone locally

@findepi Sure, i have fixed the build error, sorry for the capitalization mistake. Please review again when you have time, thank you~

[hackeryang]

Hello @findinpath , can you please cc when you have time, thank you

[findinpath]

LGTM % comments

[krvikash]

LGTM

[findinpath]

@losipiuk CPTAL ?

[steveloughran]

1. how effective has this change been in making the exception go away?
2. has anyone experimented with retry/recovery on the exception being raised?

we've had reports of this coming through the S3A connector and as well as adding a TTL, I'm thinking of classifying the the "no response" exception as a connectivity failure which we can treat as idempotent and retry on, always.

[hackeryang]

1. how effective has this change been in making the exception go away?
2. has anyone experimented with retry/recovery on the exception being raised?

we've had reports of this coming through the S3A connector and as well as adding a TTL, I'm thinking of classifying the the "no response" exception as a connectivity failure which we can treat as idempotent and retry on, always.

Hello, good question~
1、It has been for several months after we added the parameter to one of our production cluster with 35 nodes, the timeout error has gone.
2、May be the 'Fault Tolerant Execution' mode can meet your needs: https://github.com/trinodb/trino/wiki/Fault-Tolerant-Execution
https://trino.io/docs/current/admin/fault-tolerant-execution.html
And also you can set this parameter in your config.properties: exchange.data-integrity-verification=RETRY, here is the relevant pr: #3438