Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

S3 client tries to connect to http service even if endpoint_url specifies full url with https #4252

Open
swapnilsky opened this issue Sep 2, 2024 · 1 comment
Labels
bug This issue is a confirmed bug. documentation This is a problem with documentation. endpoints p2 This is a standard priority issue s3

Comments

@swapnilsky
Copy link

Describe the bug

While creating client for s3, I am giving full endpoint url like https://<service-external-ip> and use_ssl as False.
according to the Documentation,
when endpoint_url specifies http scheme, botocore will ignore use_ssl.
But this isn't happening, The created client tries to connect to http://<service-external-ip> and fails.

Expected Behavior

Client should connect to endpoint_url as is when http scheme is given such as https://<service-external-ip>

Current Behavior

Client tries to connect to http://<service-external-ip>

Reproduction Steps

`
import boto3

use_ssl = False
endpoint_url = "https://" // this is dynamic, we load this from ENV VARIABLE
s3 = boto3.client("s3", aws_access_key_id="aws-access-key", aws_secret_access_key="aws-secret-access-key", use_ssl=use_ssl, endpoint_url=endpoint_url)
lock_config = s3.get_object_lock_configuration(Bucket="bucket_name")
`

Possible Solution

No response

Additional Information/Context

No response

SDK version used

1.17.99

Environment details (OS name and version, etc.)

ubuntu 20.04, python 3, boto3 1.17.99

@swapnilsky swapnilsky added bug This issue is a confirmed bug. needs-triage This issue or PR still needs to be triaged. labels Sep 2, 2024
@tim-finnigan
Copy link
Contributor

Thanks for reaching out. I can reproduce this issue, for example:

import boto3
from botocore.config import Config

s3 = boto3.client("s3", endpoint_url='https://test.com', use_ssl=False,
                  config=Config(connect_timeout=1, retries={'max_attempts': 1}))
lock_config = s3.get_object_lock_configuration(Bucket='bucket_name')

Results in trying to connect to http://test.com/bucket_name?object-lock.

Relevant code can be found here. So regarding the documentation, the issue is with this wording for endpoint_url:

You can specify a complete URL (including the “http/https” scheme) to override this behavior. If this value is provided, then use_ssl is ignored.


I should also note that 1.17.99 is very old (the latest version of Boto3 is 1.35.12), so I recommend updating to a more recent version.

@tim-finnigan tim-finnigan added s3 endpoints p2 This is a standard priority issue documentation This is a problem with documentation. and removed needs-triage This issue or PR still needs to be triaged. needs-review labels Sep 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug This issue is a confirmed bug. documentation This is a problem with documentation. endpoints p2 This is a standard priority issue s3
Projects
None yet
Development

No branches or pull requests

2 participants