Skip to content

S3 appliance#4200

Merged
sougou merged 3 commits intovitessio:masterfrom
slanning:s3_appliance
Sep 24, 2018
Merged

S3 appliance#4200
sougou merged 3 commits intovitessio:masterfrom
slanning:s3_appliance

Conversation

@slanning
Copy link
Copy Markdown
Contributor

@slanning slanning commented Sep 19, 2018

This is meant to supersede #4068
and cherry-picked the commit from that.
(I just made a new PR because I think I can't push to Simon's.)

On top of that, it fixes some log output (*bucket)
and adds a flag to allow disabling SSL certificate validity check
(not sure why it's hard to fix the certificate..).

sjmudd and others added 3 commits September 19, 2018 14:45
* Add logging when using s3 for: StartBackup / ListBackups / RemoveBackup
* Added options to enable usage of an S3 appliance: Cloudian HyperStore:
      -s3_backup_aws_endpoint <host:port> (port is required)
      -s3_backup_force_path_style=true/false

  By default the s3 client will try to connect to
  <path>.<region>.amazonaws.com.  Adjusting the endpoint will allow this
  to be changed.

  Given the way the FQDN is configured the TLS certificate may not match the
  server's "base" (<region>.<end_point_address>) due to the leading <path>
  so setting -s3_backup_force_path_style=true will force the s3 client to
  connect to <region>.<endpoint> and then make a request using the full
  path within the http calls.

Signed-off-by: Simon Mudd <simon.mudd@booking.com>
Signed-off-by: Scott Lanning <scott.lanning@booking.com>
In case for some reason the SSL certificate is invalid...
this flag allows disabling the validity check.

Signed-off-by: Scott Lanning <scott.lanning@booking.com>
@sougou sougou merged commit c85e1a5 into vitessio:master Sep 24, 2018
@demmer
Copy link
Copy Markdown
Member

demmer commented Oct 30, 2018

This PR introduces a regression. The default value for the following option isn't correct:

	// AWS endpoint, defaults to amazonaws.com but appliances may use a different location
	endpoint = flag.String("s3_backup_aws_endpoint", "amazonaws.com", "endpoint of the S3 backend (region must be provided)")

As a workaround, we managed to change the setting to s3.amazonaws.com but the better change would make this default to empty string and then only set it in the s3 uploader options if present.

@slanning
Copy link
Copy Markdown
Contributor Author

Oops, I'll make a PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants