"signurl" is ignoring "use_https" and other configuration settings

[jamshid]

Seems the url generated by signurl should use the endpoint style described in ~/.s3cfg. In particular:

• use https if use_https = True
• put the bucket name in the path, not the domain/endpoint, if host_bucket == host_base

$ s3cmd --version   # installed with "brew install --HEAD s3cmd"
s3cmd version 1.6.0+
$ s3cmd signurl s3://mybucket/file.mp4 +3600
http://mybucket.storage.example.com/file.mp4?AWSAccessKeyId=1e71169c9ab10b293bda2b454bf20c35&Expires=1448001252&Signature=xcJUwTgRq5xSvTXp3K3zI3v%2Fq7g%3D

Expected:

https://storage.example.com/mybucket/file.mp4?AWSAccessKeyId=1e71169c9ab10b293bda2b454bf20c35&Expires=1448001252&Signature=xcJUwTgRq5xSvTXp3K3zI3v%2Fq7g%3D

Actual requests work fine: s3cmd uses https://storage.example.com. I'm using an S3-compatible service, here's my ~/.s3cfg:

[default]
access_key = 1e71169c9ab10b293bda2b454bf20c35
secret_key = secret
use_https = True
host_base = storage.example.com
# Using first format to force "bucket in path not Host:"
host_bucket = storage.example.com
#host_bucket = %(bucket)s.storage.example.com

[mdomsch]

The SSL certificates offered by AWS S3 (and other public cloud providers)
won't match your DNS hostname (either DNS-style bucket names, or CNAMEd
buckets). That then leaves it up to clients to determine if they want to
allow the https connection, or not. Recent python, for example, will
reject it. So I don't think there's a huge benefit to s3cmd replying with
'https' on the front. If in your situation you know http should be https
and will be accepted, you can always replace http with https in the
returned string after s3cmd is done.

I'm open to a better solution, but simply honoring use_https (which is
really for s3cmd's connections to S3, not other clients downstream of us -
this is an overload of the option) doesn't seem like the right solution.

Thanks,
Matt

On Thu, Nov 19, 2015 at 11:43 PM, jamshid [email protected] wrote:

Seems the url generated by signurl should use the endpoint style
described in ~/.s3cfg. In particular:

• use https if use_https = True
• put the bucket name in the path, not the domain/endpoint, if host_bucket
  == host_base

$ s3cmd --version # installed with "brew install --HEAD s3cmd"
s3cmd version 1.6.0+
$ s3cmd signurl s3://mybucket/file.mp4 +3600http://mybucket.storage.example.com/file.mp4?AWSAccessKeyId=1e71169c9ab10b293bda2b454bf20c35&Expires=1448001252&Signature=xcJUwTgRq5xSvTXp3K3zI3v%2Fq7g%3D

Expected:

https://storage.example.com/mybucket/file.mp4?AWSAccessKeyId=1e71169c9ab10b293bda2b454bf20c35&Expires=1448001252&Signature=xcJUwTgRq5xSvTXp3K3zI3v%2Fq7g%3D

Actual requests work fine: s3cmd uses https://storage.example.com. I'm
using an S3-compatible service, here's my ~/.s3cfg:

[default]
access_key = 1e71169c9ab10b293bda2b454bf20c35
secret_key = secret
use_https = True
host_base = storage.example.com

Using first format to force "bucket in path not Host:"

host_bucket = storage.example.com
#host_bucket = %(bucket)s.storage.example.com

—
Reply to this email directly or view it on GitHub
#666.

[zokier]

Would a configuration option be acceptable solution here? For example signurl_base = http://%(bucket)s.%(host_base)s as default which then could be changed to signurl_base = https://%(host_base)s/%(bucket)s to match @jamshid needs.

[zokier]

I just noticed #551 which is similar issue, and it mentions public_url method. Should signurl use that as the URL base instead? Then a common solution for these issues could be made.

[jamshid]

Whoa just noticed this issue number 🤘.
Yes, public_url from #551 would be good here too. In my case I'm also using an S3-compatible that provides a valid cert (maybe AWS CloudFront also handles this).
I'm having to do some sed to switch the output of signurl to https and move the bucket into the path.

[jamshid]

With the recent 2.0 changes around handling of certificates, maybe this request can be reconsidered?

Ideally s3cmd would simply adhere to the style specified by host_bucket and let use_https determine the protocol of the url it outputs.

http access is deprecated, no one will want to give out a non-https url. Generating "bucket in path" style url avoids certificate problems.

[fviard]

Interesting topic. Your suggestions are smart for handling the general case but I still think that the auto magic will be problematic with the variety of targets. so, the best solution would be something based on public_url. In fact, for that, we can probably use the "website_endpoint" configuration that is barely used. The only question is "https" or not by default. I would like to say yes, but thinking more that it would be better in "http" for basic users. Le 5 juil. 2017 9:34 PM, "jamshid" <[email protected]> a écrit :

…

With the recent 2.0 changes around handling of certificates, maybe this request can be reconsidered? Ideally s3cmd would simply adhere to the style specified by host_bucket and let use_https determine the protocol of the url it outputs. http access is deprecated, no one will want to give out a non-https url. Generating "bucket in path" style url *avoids* certificate problems. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#666 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABAUpO5OgbdPOsPkwPwfApjdlxAaAfN4ks5sK-VNgaJpZM4GmIze> .

[thelan]

Hello,
It's quite an old issue but i thkink we can close this. I've added an option to the configuration file to manage this case:
In case you want to see the PR #917

[chrisallick]

This definitely should be configurable.
https should be default.